Wire extension to show exp run in Ctrl-F5?

[shcheklein]

When you click ^F5, or in my case I can "play" it from the keyboard, I would expect to see a possibility or even run automatically.

Question: how this thing is organized? Can we control it?

[rogermparent]

The F5 and ^F5 menus are interfaces specifically for debuggers (even though ^f5 says "without debuggers", it refers to running a debugger command without hooking into the debugger).

Generic external commands that don't involve debuggers like dvc exp fit more into the Task API (ctrl+shift+b as opposed to f5). While debugging is a requested feature, we have no idea how to run a debugger using dvc to call to a Python script as opposed to just calling the Python script directly, which is likely supported by the Python extension's debugger.

[rogermparent]

I think we could actually load up dvc through the Debugger API- I just set one up locally and it works!

Unfortunately, it will involve switching how we run exp run entirely unless we maintain two different ways to invoke exp run, which is obviously not preferable. We should be able to invoke a debugger-based exp run through all the same places, however, assuming there's a programmatic way to invoke a specific debugger (which I admittedly can't confirm but it would be really odd if there weren't)

[rogermparent]

After playing around with a debugger-based exp show runner, it is possible to do so but doesn't run as a debugger because dvc is being used as an intermediary- dvc itself is being debugged and we can see the threads it spawns, but breakpoints do not work because the train.py script is external.

Also, the stop functionality of the debugger UI stops with a signal DVC doesn't play nice with- either SIGKILL or SIGHUP, not SIGINT. This causes the "there's a DVC process still running, remove .dvc/tmp/rwlock" error.

I think there's too much stopping us from actually taking advantage of this API, but if we can work around the issues (particularly the stop signal issue) then it could be a viable path for the future.

Here's a video showing off what I tried out, both with success and the rwlock issue at the end:

debugger-exp-show-demo.mp4

[shcheklein]

The idea we discussed is also to see how Jupiter integrates with VS Code and its actions.

[mattseddon]

We might revisit this in the future but for now we are going to park this.