Fix debug mode API

[plafer]

Making a debug.stack instruction to work is currently much harder than it needs to be. I can never remember all the places I need to go modify to get it to actually work. My workflow while debugging a test is typically:

1. add a debug.stack instruction, notice that it doesn't work
2. Change the line that creates the Test struct from e.g.

let test = build_test!(source, &stack, advice_stack, store, advice_map.iter().cloned());

to

let mut test = build_test!(source, &stack, advice_stack, store, advice_map.iter().cloned());
test.in_debug_mode = true;

Notice that this doesn't work either, and remember that the VM probably is built with debug mode off.

3. Dig down the Test::expect_stack() call stack to where I find the processor::execute() call, and change the line from

processor::execute(
            &program,
            self.stack_inputs.clone(),
            &mut host,
            ExecutionOptions::default(),
        )

to

processor::execute(
            &program,
            self.stack_inputs.clone(),
            &mut host,
            ExecutionOptions::default().with_debugging(),
        )

Notice that this doesn't work either.
4. I look for the line that actually calls Host::on_debug(), which is in Process::execute_decorator(), and comment out the self.decoder.in_debug_mode(). But that still doesn't work.

Conclusion

Clearly I'm missing something maybe "obvious", but every time I spend way too much time fiddling with the assembler and VM to have a debug.stack instruction go through. And I work on the VM everyday - I imagine anyone who doesn't simply gives up after a while.

I haven't thought about how to fix it, but enabling debug mode should be very easy to do. As I understand, the constraints are that

1. on the assembler side, we don't always want to generate debug symbols, as this adds considerably to the size of the binary
2. on the VM side, we don't always want to execute the debug decorators for performance reasons when we don't care about them, even if they're in the binary (i.e. avoiding a bunch of prints)

I'm sure however that there's a way to have a single "debug on" switch that enables everything properly. Some extra nice-to-have's would be:

1. if the assembler catches a debug instruction, but is currently not in debug mode, it could output a warning.
2. Similarly on the VM side, we could output a (single) warning when we catch a debug instruction when not in debug mode

• this is unlikely to affect production anyways, since we expect production binaries to be compiled without debug symbols

[plafer]

Additional concern when debugging a test: adding a debug.stack to an stdlib file will not work, because the stdlib is not compiled with debug symbols by default.

A potential fix for this is to compile stdlib with the with-debug-mode feature for test targets.

[bitwalker]

Personally, I use the debugger I built for debugging programs emitted by the compiler. Once miden-package is landed here, we should be able to move midenc-debug into this repo, or perhaps merge it into the miden executable, and make it the debugger. It provides a ton of useful information and capabilities, including source locations, but is currently tied to the compiler due to packaging.

The compiler also defaults to enabling debug mode in the assembler during codegen - it must be explicitly disabled, because not having debug information makes the resulting program impossible to debug. It also links in debug-compiled versions of the standard library, etc.

The debugger implements its own executor in order to capture additional information about the program, handle specific trace events, and to allow stepping back/forward, etc., and I think that's pretty much a requirement for any half-decent debugging experience.

All the tools needed are basically there, they just have to be assembled together into something cohesive, and I think midenc-debug is probably as good as it gets at the moment. I'd like to further improve it, and address some small issues/annoyances, but haven't really had the time, and figured it'd be best to wait until we can integrate it into the larger Miden toolchain.

[bobbinth]

A few thoughts on how we can potentially simplify this:

• We could remove the "debug" option from the assembler entirely. That is, the assembler would always assembler MAST forests in debug mode, and then the debug info could be either truncated separately as needed, or maybe when libraries etc. are loaded, we could load them either with debug info or without (the way we serialize MAST forests should make this relatively simple).
• Instead of using runtime debug flags, we could transition to compile-time features. That is, both stdlib and the VM itself would have something like a with-debug-mode feature. When enabled, MAST forests would be deserialized with debug info, VM processor would process debug decorators etc. This does make this a bit less flexible, but I also don't know if we'd ever need to run the same compiled binary with VM in both debug and non-debug modes.