Show HN: A Ghidra extension for exporting parts of a program as object files

It is definitely not going to be easy to do for every CPU architecture, but it's not as ridiculous as it seems. Basically, the difference between an "object file" and an executable file or shared library is not very large. Often times, except for Microsoft platforms, they are in the same actual file format, e.g. ELF.

The big thing is relocations. Object files have granular relocations that executable files don't; at least on Windows, executable images just have minimal relocations that point to addresses of code and data that will need to be fixed up if the executable is relocated, but the executable image itself is only able to be relocated with regards to its image base address as a whole unit. In contrast, object files contain symbol-level relocations. To be able to accurately reconstruct this information, you need to annotate the disassembly with somewhat accurate information about symbols.

The other big difference with an object file is well, it is not linked. None of the symbols are "resolved". This is particularly easy to fix actually: during delinking if the symbol is outside of the current scope then it just needs to be replaced with an unresolved symbol, pretty much. Then when relinking, another object file or library needs to provide the symbol so that it can be linked back up.

(There are a few smaller differences too, like the lack of an entrypoint, but it really isn't a whole lot of significance.)

This means that the boundaries for which you carve object files out of an executable image or shared object is actually completely arbitrary. It obviously was segmented into translation units when it was originally compiled, but nothing really cares about those boundaries at the linking stage. (Of course, you probably want to try to figure it out if possible, since it will probably be very hard to do a matching decompilation if your object boundaries are incorrect.)

Take this with a grain of salt as despite having literally worked on this problem I feel like I might be messing up some of the details a bit. I've been meaning to write a blog post about object files, though there are actually a couple of good ones already floating around.

To keep things short, object files are made of three parts: relocatable section bytes, relocation tables and a symbol table.

When a linker is invoked to generate an executable from a bunch of object files, it will lay out their sections in memory, compute the addresses of the symbols in the virtual address space and apply the relocations based on the final addresses of the symbols onto the section bytes.

The trick to delinking is figuring out where those relocations were applied in order to undo them and get back relocatable bytes. Then, you create relocation tables based on what you've just undone as well as a symbol table, package it all and you'll get an object file.

The really tricky part is the analysis for spotting the relocation spots. I'm leveraging Ghidra to do the bulk of the work, but it still requires some work to convert references into relocation spots (fairly easy on x86, nightmarishly difficult on MIPS) as well as collecting all the required data and serializing the object file itself, hence this extension to automate all of that.

Actually there are a few attempts at this! Here's one for Microsoft Program Database:

https://github.com/wbenny/pdbex

As for using an LLM to tidy it up... It doesn't seem like there has been a ton of success applying LLM models to reverse engineering yet... A part of me is wondering if this will wind up being a place where the LLM architecture proves insufficient. I'm not an expert but if I had to place a bet I'd bet on diffusion models being more interesting for a lot of reverse engineering use cases. That said, it's not really the same thing, but with Binary Ninja they have a feature called Sidekick that uses an LLM to try to clean up the disassembly; I'm kind of unimpressed but maybe it is useful to somebody.

I wrote a tool a few years ago which automatically generated and inserted type-aware fuzzers for C APIs from DWARF info: https://github.com/intel/fffc

Generating headers and also mutators that you could then modify to meet type constraints was part of that.

Edit: just to add onto the LLM side, I can see this labelling anonymous structs or similar but I'm not sure that's a good idea. What might be interesting would be to try to get an LLM to verbalize/summarize known type constraints for documentation purposes.

pahole gives you compilable C header files from ELF DWARF information. LLMs seems irrelevant here: either your header files have all the types exported from the executable correctly so they are usable with the original values, or they aren't correct/complete and having an LLM make up some more doesn't help.

Ghidra also has native functionality to export its data structures, which it can create from DWARF structures (Right click -> Export to C header).

Tangentially, I've considered generating debugging symbols for the exported object files, based on the contents of the Ghidra database, in order to improve the debugging experience when using them.

I haven't implemented that feature yet because so far I've managed to get by without it. Also, it sounds like a rather deep rabbit hole to fall into and the one I'm currently inside of is big enough as it is.

How much work is it to figure out which sections of the executable to export?

As long as you do not cut across a variable or a function, you can export pretty much however you want, you don't have to follow the original object file boundaries. What to export is a separate matter and requires some knowledge about the program. Having debugging symbols makes this much easier, otherwise by the time you've made the Ghidra database accurate enough for exportation you'll usually have an idea of where's what.

Would it be realistic to be able to export a modern-ish (2008-2015) Win32 game into objects and then compile/link it into a full executable again with less than a few hours work?

About the user report in my submission, they first raised an issue in early July and by mid-August they got a fully working, functionally identical relinked executable. To be fair, the COFF exporter had a lot of bugs that needed to be fixed and the i386 analyzer needed some touch-ups, things that somebody else should hopefully won't stumble over now.

I don't know how long it would take, but unless you have debugging symbols and are really lucky it will take more than a few hours of work. A skilled reverse-engineer can probably manage to get something executing in that timeframe (even if it crashes halfway during the first loading screen), but it's one of these tasks that you won't know when it will be done until it is done.

Based on what it seems that you're asking, it is not, and cannot be, a foolproof process. consider

  int getSpecialArrayElement(char *array, uint64 key) {
     i = computeOffset(key);
     return array[i];
  }

So is this a completely fool-proof process?

That's... complicated to answer.

My analyzers rely on an accurate Ghidra database, at least for the parts you want to export. While I've put a fair amount of effort into logging the various issues than can crop up which require fixing, they can't see what isn't there. In particular, missing references and truncation of variables won't be detected and will result in exotic undefined behavior.

There are ways to track down some of these issues. The best I've found so far is to relink the executable at a different base address and making sure that the original program's address ranges are unmapped ; that should lead to segmentation faults when absolute relocation spots are missed that can be debugged (but that only works if your target has a MMU). Truncated variables are very tricky to troubleshoot (especially if you don't suspect it) since it's the memory following the truncated variable that gets corrupted. An integer that is mistaken for a pointer can also be very tricky to track down, as the integer's value will vary depending on the address the target symbol gets, leading to erratic program behavior (that's especially an issue for a program loaded very low in the address space).

That being said, if the Ghidra database is accurate enough and you export back to the same object file format used originally and you subsequently use it onto the same platform with the same toolchain, you can delink megabytes of program code and data successfully. I consider that if the linker did it, then it should be possible to undo it.

Now, if you start cross-delinking to something that doesn't match the original program's platform and toolchain (like delinking from a Linux i386 ELF executable into a COFF object file and using it with a i386 Windows toolchain) then it's another story. If the exporter can express the relocations then you might end up with a working relocatable object file, but you'll still have potentially mismatched ABIs to contend with. It can be done, but that's not something I would recommend as a first project.

TL;DR Depending on what you do and the accuracy of the Ghidra database, it can range from "it just works" all the way to praying to Cthulhu for mercy.

Thanks!

I should get back to this project one of these days. I did one version tracking session too many in a row and had to take a break, plus that delinking side-quest keeps snowballing out of control.

There are links to various case studies on my blog inside the README of the repository.

https://github.com/boricj/ghidra-delinker-extension/blob/mas...

"... past life ..."?

Are you some kind of a ressurectionist?

I'm actually a pretty big n00b when it comes to Ghidra and haven't even really figured out a good way to select the ranges for delinking from a large binary.

One way to keep track of things to delink is to use folders and fragments inside a program tree. For example, I have a Ghidra program where I've figured out the name and ranges of the various object files that originally made up the executable. These folders or fragments can then be selected as a whole with right-click > Select Addresses.

The relocation synthesizer analyzer and the exporter can also be scripted, either independently or using the program's tree manager. This removes the need to select by hand the ranges you want as well as invoking manually the analyzer and the exporter.

While objcopy can do many things, it can't undo the work of the linker. If relocations aren't unapplied and a new relocation table generated, these spots inside the new object file will reference the original program's address space, leading to some exotic undefined behavior.

Delinking is a subject with very few resources online, but there are a couple of other tools for it out there:

  - https://github.com/endrazine/wcc

  - https://github.com/jonwil/unlinkerida

  - https://github.com/jnider/delinker