How notebook debug cell works
Notebook cell debugging is similar to how python debugging works. However it uses a DebugAdapter instead of a DebugAdapterDescriptor.
There's a number of classes involved. This sequence diagram describes how they interact:
These different steps are described in detail below:
The first step is the user clicking the 'Debug cell' button.
The command handler then generates a launch configuration entry that looks something like so (it's not literally json, but rather a struct in memory):
{
"type": "Python Kernel Debug Adapter",
"name": "mynotebook.ipynb",
"request": "attach",
"justMyCode": true,
"__mode": "debugCell",
"__cellIndex": 2
}where
- the "type" relates to a registered DebugAdapterDescriptorFactory that generates DebugAdapterInlineImplementation objects.
- the "request" is an attach
- it has extra properties indicating which cell and whether or not it's debug cell or run by line.
The debugging manager then creates a Debugger with this configuration.
The DebuggingManager just starts debugging with the configuration. This switches VS code into debug mode.
Because a DebugAdapterDescriptorFactory was registered and its type was passed in the launch config, VS code will then attempt to create a DebugAdapterDescriptor.
The KernelDebugAdapter is created as a result of this call and passed as the implementation to the DebugAdapterInlineImplementation
The KernelDebugAdapter is an in memory server that is supposed to listen to DAP messages. For more information on handling DAP messages, see the description of Python debugging.
The KernelDebugAdapter is shared between cell debugging and run by line, the only difference being the controller. The controllers job is to intercept certain messages from VS code (and certain responses from debugpy) to force the behavior necessary for the mode it is in. For the DebugCellController, this means behaving just like ordinary python debugging.
After the KernelDebugAdapter and DebugCellController are created, the DebugAdapterInlineImplementation is returned to VS code. It will now use it to send all DAP messages to control the debug session.
The first DAP message is Initialize.
The KernelDebugAdapter's handleMessage is called with the contents of the Initialize message.
The KernelDebugAdapter then parses the contents of the message and translates the cell URI's in it to the expected file name paths debugpy will be seeing in IPython. This has to happen on every request (well because each request has the cell URI and not the paths the debugger expects)
How does it know the file paths? IPython has a new special message - dumpCell. This takes the contents of a cell and generates a temporary file for it. That temporary file is used as the translated file path.
Why not just translate the paths in the message to the <ipython-hash.py> format? This has a couple of advantages:
- Stepping into other cells from the point of the view of the debugger is just another file.
- dumpCell is on the remote machine so file paths are always local to the kernel
- ipykernel didn't need a custom message so that debugpy knew how to map files
The Initialize message is then sent to IPython by making the jupyter requestDebug call.
Now you might wonder did IPython implement its own debugger?
No, it actually loads debugpy into itself and forwards DAP requests to debugpy.
Since the requestDebug is just a normal jupyter message, the response is handled just like an execute.
At this point the KernelDebugAdapter has the response from IPython. It then gives the DebugCellController the chance to swallow the message (or send a different response). For Initialize the message is ignored.
The KernelDebugAdapter then fires an event for VS code to handle that contains the response.
This same pattern is used to handle requests and responses until debugging is setup.
Except for the ConfigurationDone message. When this message comes back from the debugger, it means debugging is established. VS code is now 'attached' to the kernel.
This is when the DebugCellController intercepts the message and tells the kernel to start executing a cell.
At some point debugpy will need to send an event to VS code to indicate the process has stopped. Since there's no 'event' mechanism in the JMP, an out of bound message is delivered on the IO Pub channel.
Just like with the responses to DAP messages, the DebugCellController is given the chance to swallow or change the stopped event. This is where the RunByLineController (if it was used here) would make a different choice and alter the behavior of the stopped event.
The stopped event is forwarded onto VS code, indicating that the debugger should stop. After this event is received in VS code, just like with Python, VS code will then ask the KernelDebugAdapter for variables, stack frames, threads, and modules.