Cross-platform debugging

December 2018

I have been meaning to write this article for some time now. I had notes hanging around for at least one year, and recently proved to be useful once more. I am sharing these in the hope that they will become useful for others as well.

This article shows how to use gdb to debug a binary for architecture X (e.g. mips) when you are using a machine with architecture Y (e.g. amd64).

qemu-user

QEMU comes mainly with two sets of binaries called qemu-system-* (Debian package qemu-system) and just qemu-* (Debian package qemu-user). The former binaries can be used for full system emulation (i.e. including hardware, mainly for virtual machines), whereas the latter can be used to "translate" userland (i.e. translate instructions to the host architecture and convert syscalls). It is clear that for simple binaries, qemu-user is the way to go here.

A very nice tutorial can be found in this reverseengineering.stackexchange.com page. The general idea is to install the required QEMU packages (Debian offers qemu-user and qemu-user-static), together with helper packages qemu-user-binfmt, binfmt* (or directly binfmtc).

On top of this however, some binaries may require additional libraries (e.g. musl libc). You could just get away with installing the Debian package for the required architecture. This is done with the a dpkg multiarch setup as explained in the Debian wiki.

At this point, binaries can be executed either by running them with QEMU (e.g. qemu-user-mips ./binary) or directly as a normal binary (e.g. just ./binary).

If you still get errors for missing libraries which you have installed, it may be that the binary loader searches them in a different path. You might also get some weird SEGFAULTs, or just strange error messages that do not make sense. The easiest way to debug such issues is to run the binary using QEMU and pass the -strace flag. In this case you get a detailed output of the actions performed, together with clear paths from where libraries are loaded, unsupported syscalls, reasons for SEGFAULTs.

Lastly I want to mention that apt-file takes the -a / --architecture flag to filter the search.

gdb

Debugging the binary with gdb is the next step. The first thing to note here is that usually the gdb binary which is distributed with a standard gdb package does not support other architectures. For example if I try to set a different architecture I get the following message:

gef ▶ set architecture mips
Undefined item: "mips".

To get arround this, Debian (and probably other distros) have either architecture specific builds of gdb (e.g. gdb-arm-none-eabi) or the gdb-multiarch package. I personally prefer the latter. Now instead of running the regular gdb binary, you instead run gdb-multiarch. All your scripts and configs should work automatically, and there is almost no change in your usual workflow.

gdb however cannot run the binaries directly, so the idea is to run the binaries with QEMU and pass the -g flag. With this flag, QEMU acts as a remote debugging server for gdb.

qemu-mips -g 1337 /path/to/mips-binary

qemu does not actually start the binary at this point. The process is waiting for the debugger to attach. The final step is to run gdb-multiarch and execute the following commands:

set architecture mips
file /path/to/mips-binary
target remote :1337
continue

Done. Also, check out this article that I used as inspiration.

Table of contents