gdb tracee io hacks

March 2017

Recently I found myself in the situation where I wanted to debug/explore an ncurses based program. The program in question is vifm, a file manager inspired from the vim editor. My problem was that I wanted to interact with the program and use gdb at the same time. Since this was an ncurses based program, it was not easy to do both things in the same shell.

With the power of the internets, I was able to find three solutions to this problem. I will explain all three and give some references for further reading.

The gdbserver method

The first and probably easiest solution is to use gdbserver in a different shell. The idea comes from this answer on StackOverflow. The answer is given to the exact same question that I have. The person asking the question was trying to use the tty command in gdb, but the target shell was "eating" the input before it would be sent to the traced application.

By using gdbserver, the shell would temporarily exit its usual interactive mode and would forward user input to gdbserver, which in turn would forward it properly to debugged application. Problem fixed.

The tty method

The second method involves the tty command in gdb. This command takes on parameter, namely the path of the tty in the /dev folder. In order to find out the value of the argument, run in the target shell (the one where you will interact with the tracee) the command tty. Then in gdb type tty <previous output of tty>. These steps have to be performed before the run command. So far this is pretty much what the guy asked above on StackOverflow.

In order to fix the issue that he was having with the input not being forwarded by the shell to the application, we need to stop the interactive mode temporarily. The easiest way to achieve this that I could think of was to start an infinite loop in the target shell. I just run this code which works for zsh:

while [[ 1 ]]; do; done

In case you are using bash, the above snippet will not work because bash does not like empty loops. The solution is to use the nop-hack:

while [[ 1 ]]; do

At this point, you can type the run command in gdb and start debugging.

The dup method

This method is partly based on this small article and its comments. It was very interesting for me to learn that I could call functions directly from gdb within the context of a running process. It was also interesting to learn about the dup system call in linux.

This method is similar to the tty method for the fact that the target shell needs to execute the empty infinite loop. Once this is done, in gdb you need to run the following commands.

b main
call close(0)
call close(1)
call close(2)
call open("<previous output of tty>", 2)
call dup(0)
call dup(0)

The commands above put a break point for the main function and start the process. When the break point is hit, we close the stdin, stdout and stderr file descriptors. We open a new file with a call to open. The second argument to open is the constant 2 which opens the file for both reading and writing. Because of the way open works, the new file descriptor will be 0. The calls to dup then duplicate the new file descriptor for stdout and stderr. As a result, all three channels will go through the target shell.