Please explain the meaning of these Debug in GDB via ssh errors.

[serby2000]

I use the last version of Ghidra built from the source (10.1 DEV) with Java 11 on Windows 10. The target machine is running Ubuntu 18.04 LTS with GDB 8.1.
When I try to debug a simple test binary using Debug in GDB via ssh I get errors like:
Resync: Missed loaded module/library: [followed by the path and the name of my binary]
Resync: Missed unloaded module/library: [followed by the path and the name of my binary]
Could not notify inferior started
Could not update stack <GdbModelTargetStack: path=Inferiors[1].Threads[1].Stack model=agent.gdb.model.impl.GdbModelImpl@443d9d19 schema=Stack> on STOPPED
Also the Dynamic , Stack and Registers windows are always empty.

I would be really grateful if smb could explain the meaning of these errors to put me on the right path. The rest I will figure from the Ghidra debug module source code.

Thank you

[d-millar]

Will take shot at the errors tomorrow, but I’m not sure they are directly related to the missing information. As you may already have surmised, the only two windows connected directly to the target are the Objects view and the Interpreter. Everything else is mediated by the active trace. So, the main question along those lines is “Is there an active trace?”.

The first thing I would check is whether the Objects view is open to the current thread. If it isn’t open past the process, the recorder may not have triggered. If you have any threads in the Thread view, then you have an active trace.

If you have an active trace, the next thing I would check is that one of the threads in the Thread view is selected. If nothing is selected there, the Registers and Stack will be empty, as well the Dynamic view (although the Dynamic view can be empty for other reasons).

I’ll stop there for the moment, but happy to help you trouble shoot this. We haven’t had too many folks trying out the ssh connection, so am interested in getting it up to speed.

[serby2000]

I do get an entry in Inferiors (it's always called 1 - < null >) and the Threads branch is populated with the proper Thread. The thread is automatically selected but the listed windows are still empty.
This error is always thrown after starti is executed:

Command completed with <GdbCommandDoneEvent {} >, not any of [class agent.gdb.manager.evt.GdbCommandRunningEvent] java.lang.IllegalStateException: Command completed with <GdbCommandDoneEvent {} >, not any of [class agent.gdb.manager.evt.GdbCommandRunningEvent]
at agent.gdb.manager.impl.GdbPendingCommand.checkCompletion(GdbPendingCommand.java:234)
at agent.gdb.manager.impl.cmd.AbstractLaunchGdbCommand.complete(AbstractLaunchGdbCommand.java:52)
at agent.gdb.manager.impl.cmd.AbstractLaunchGdbCommand.complete(AbstractLaunchGdbCommand.java:22)
at agent.gdb.manager.impl.GdbPendingCommand.finish(GdbPendingCommand.java:71)
at java.base/java.util.concurrent.CompletableFuture$AsyncRun.run(CompletableFuture.java:1736)
at java.base/java.util.concurrent.ThreadPoolExecutor.runWorker(ThreadPoolExecutor.java:1128)
at java.base/java.util.concurrent.ThreadPoolExecutor$Worker.run(ThreadPoolExecutor.java:628)
at java.base/java.lang.Thread.run(Thread.java:834)

Build Date: 2021-Sep-12 0845 EEST
Ghidra Version: 10.1
Java Home: C:\RedHat\java-11-openjdk-11.0.10-1
JVM Version: Red Hat, Inc. 11.0.10
OS: Windows 10 10.0 amd64
Workstation: NENIAC

After taking a look at the source code I think that the binary opened from the local Windows machine is not recognized as being the same with the one debugged by GDB on Linux but I am not sure.
I want to understand the meaning of those errors to find the problem myself but I don't have the time to review everything. If I know what was supposed to happened I can save a lot of work.

[d-millar]

Just an update: on the good news front, I can completely reproduce your situation; on the bad news front, not sure what's happening. Working on it - will keep you posted.

[d-millar]

OK, we understand the problem - just need to figure out a solution. Apparently, if you connect via ssh from a Windows box to a Linux target, the target sends the response to every MI2 command, plus an additional two lines that we were not expecting: "&\n" and "^done". The first is not a problem, but the second is completing commands that are in the queue which haven't received a legitimate response yet. Hopefully, a solution tomorrow....

[serby2000]

I immediately knew what was wrong when I saw that "&\n". GDB adds that empty log when you send a bad character at the end of a command line. You are using println in the output stream and println in Windows adds both *\r and \n but GDB expects just \n .
That is the old trap for young Windows players. All Ghidra code should be scanned for such errors.

Thank you very much! You saved me a ton of time

[d-millar]

Thank you - both for pointing out the bug and the solution! You’ve definitely helped us make the debugger better!

[nsadeveloper789]

I went ahead and created an issue for this. I wound up restating the things you both already figured out here, because I apparently have trouble reading before writing :) . Thanks again for your help.
I don't think we need to go about scanning all of Ghidra for this, though. The case where Ghidra is running on one OS and communicating with a remote system of another OS using a text-based protocol is unique to this GDB/SSH use case, AFAICT.

[serby2000]

Thanks for the support. I am working on some Debugger enhancements. At the moment I'm piecing together the architecture. I'll ask if I get stuck again.

[astrelsky]

I am glad I could be of help and thanks for such an excellent piece of software!
A little more documentation on the inner workings and the architecture of Ghidra and the debugger would be really helpful for those that want to contribute in a meaningful way to this project. Even the API documentation is on the thin side.

ps on a teeny tiny portion (like 10%) of the total documentation is provided in the distribution and build when building from source. Not that this has ever stopped me from building them all. I just wouldn't hand them out because most of them aren't built for a reason and people who use those api's will end up shooting themselves in the foot sooner or later.

These docs?

I also considered implementing a gadp but after looking at the docs, some examples, references and a wall of classes I needed to implement I decided it would be much easier to implement a gdb stub for my target.

[d-millar]

@astrelsky Was the reference to Windows docs just an example or is there a piece of the API you’d like added to the Ghidra debugger? Is the target you’re implementing a gdb stub for off general interest?

[astrelsky]

@astrelsky Was the reference to Windows docs just an example or is there a piece of the API you’d like added to the Ghidra debugger? Is the target you’re implementing a gdb stub for off general interest?

That was just an example of documentation not normally built. Just happened to grab something near the top of the packages list.

As for the target, there may be some interest in it but it is not something I would expect the Ghidra team to handle. It is similar in nature to what is being done in #2721. I suspect there will be a lot of things of this nature.

[d-millar]

Gotcha - well, if we can be of assistance, definitely let us know. I do think you are correct re implementing a gdb stub being the easier path, having done two of these now myself. :)