Explore improving the single file debug experience

[weinand]

Today VS Code debugging has no generic way to know whether a debug extension implements "single file debugging". Without this knowledge it is not possible to provide a standardised UX experience for this case.

Because of this some debug extension have started to create their own UI. A good example is the Python extension which shows a play button in the editor:

[weinand]

How "single-file debugging" works today:

• precondition:
  • no launch.json exists in workspace
  • program is open in current editor
• based on language type of current editor all debug extensions are searched that are configured for that language (languages property in the debuggers contribution).
• if more than one is found, a Quickpick is used to let user pick one debug extension
• the extension is activated
• if the extension implements resolveDebugConfiguration, it is called with an empty in-memory launch config
• If the debug extension supports "single-file debugging without launch.json", it understands the empty in-memory launch config and "upgrades" it, so that it starts debugging on the file that is open in the editor.
• if no extension can be found or if an extension doesn't support the "single-file debugging without launch.json" nothing happens.

@isidorn is that a faithful description of the current behavior?

[isidorn]

@weinand yes this is faithful description of the current flow.
Just a nitpick: if no extension can be found we do not open launch.json. We will only open launch.json if an extension explicitly tells us to do that by returning null.

[weinand]

What are the problems/paint points that we try to address?

A first pair of problems is related to the fact that VS Code behaves differently if the workspace has a "launch.json" or not:

• Assuming that no "launch.json" exists in the workspace, VS Code does not know (upfront) whether a project (workspace) is debuggable, e.g. whether pressing a play button would successfully start a debug session. Corollary: VS Code does not know the reason why a project (workspace) is not in a debuggable state either. These problems makes it difficult to offer a robust "Run" button in the Welcome view. If the debug extension does not support "single-file debugging without launch.json", pressing "Run" does nothing (and we do not really know why and cannot help the user).
• If a launch.json exists, VS Code assumes that the user uses it exclusively (by selecting a launch config and executing it) and no smartness is employed. Even if a debug extension implements the "single-file debugging without launch.json" scenario, it is no longer available in the UI. That's the reason why the Python debug extension adds a "Play" button to the editor: with this button a user can quickly and easily override a currently selected launch config and start debugging on the file open in the editor.

Another problem comes from the fact that VS Code does not want to activate debug extensions eagerly. As a consequence VS Code can only rely on the declarative information available in the package.json but not on running extension code.

Today VS Code only knows the supported languages of a debugger. This information is used in the "single-file debugging without launch.json" scenario for activating a debug extension based on the language open in the current editor. But this mechanism does not work if a file with an unsupported language is open in the editor or if there is no file open at all. Running extension code could easily figure out that the project (workspace) is runnable/debuggable. E.g. it could determine the entry-point of the project by analysing the package.json. (Actually this code exists already in many debug extensions but it cannot be used without activating the debug extension).

"Run without debugging":
DAP defines protocol for "Run without debugging" and VS Code implements it, but not all debug extensions are implementing it.
Since there is no way to figure out whether a debug extension implements the "Run without debugging", VS Code always provides UI for "Run without debugging" even if the debugger doesn't support it. Typically this is not a big problem because a debug extension will fall back to "Debug" if "Run without debugging" is not implemented.

---

Some proposals (in no specific order):

• Introduce explicit extension API (e.g. provideWorkspaceDebugConfiguration) to have the debug extension communicate back to VS Code the following information:

  • whether debug extension supports a "Run/Debug" in the current situation (without relying on launch.json or relying on an active file in the editor)
  • if the debug extension cannot figure out what to run/debug, it should report back a message explaining why run/debug doesn't work and what to do.

  VS Code would use this API for the Welcome view and "F5" (if no launch.json exists and there is no active file). E.g. in the a Node.js this API could find the program's entry point in the package.json. Today the resolveDebugConfiguration is able to launch the program in that case but we are not able to show a Play button if the user has the package.json open in the editor.
  If "run/debug" is not possible the welcome view could show how to fix this.

• Introduce explicit API for running/debugging a file (or resource?), e.g. provideFileDebugConfiguration or provideResourceDebugConfiguration. VS Code would use this API for supporting a "Play" button in the editor.

• Introduce new API for determining whether the DA supports the "noDebug" mode (surfaced as "Run without debugging"). I'm not sure whether it is sufficient to add this to the DAP or whether this must be extension API or just a new attribute on the "debuggers" contribution (dependent on the location of the API the info would be available statically, dynamically on extension activation, or dynamically on debug session start).

• the problems resulting from the fact that VS Code tries to avoid activating debug extensions eagerly are more difficult to solve. Possible approaches:

  • activate at least one debug extension eagerly based on historic information from previous debug sessions. We have this information already as a "when" context.
  • try to match an active language extension automatically with the corresponding debugger extension. This is based on the assumption, that a beginning user starting a Python project most likely needs a Python debugger.

How to design the APIs?
Today some of the debug extension APIs revolve around "debug configurations" (provideDebugConfigurations, resolveDebugConfiguration, startDebugging).
To continue this "style", I propose that new APIs follow suit:
instead of introducing a new API for "start debugging a workspace/project" or "start debugging a file", we could introduce a new function provideDebugConfiguration(options) that based on the passed ´options´ returns a debug configuration (or an descriptive error if debugging isn't possible). The debug configuration can be passed to startDebugging to actually launch a debug session. Since provideDebugConfiguration(options) does not start the debug session itself, it can be used for probing whether debugging is possible.

[isidorn]

Good analysis of our current limitations. I can brainstorm on potential ideas on how to solve some of these, but you might be already writing this, so let me know once you are looking for feedback. Thanks!

[weinand]

@isidorn I'm done with my proposals. Please review and provide feedback. Thanks.

[weinand]

Discussion will be continued in April.

[isidorn]

Good writeup. Some feedback:

Play button

In order to know if VS Code should put a Play button in the editor title VS Code should ask all activated debug extensions if they support a Play button for that file. If any of them do, the Play action should be there. The extensino can not decide this based on VS Code api, but VS Code debug UX would have to pass the file / uri for who it is asking. Because there can be inactive editors opened side by side.
This could be done by your proposel of provideDebugConfiguration. VS Code would simply call this for any visible editor for all adapters, and if there is 'candidate' we would show the play button.

Extension activation

I do not see the issues of extensions not being activated. If the extension is not activated then 99% it can not handle the open file, since it would already activated on that file type.
To make this discussions more scoped and tue to the previous reason I suggest to handle the issue of extension activation seperatly.
If you think we should solve this with this issue what would be the example scenario where the extension is not activated? The only scenario I see is I just opened a workspace and I did not open a file, I just want to press F5 to run. However that seems unrelated to the single file discussion.

Run without debugging

This sounds best fitted for a static contribution point via package.json imho. However if a debug extension says no to this how would that change the VS Code debug ux experience. Just some actions would be disabled?

fyi @connor4312 @roblourens

[weinand]

@isidorn thanks for the feedback.

Play Button:
"should ask all activated debug extensions": yes, but the "activated" is already a problem: if a user has created a new Python file, the debugger is not activated and no Play button will be shown. This can be solved by either adding some static contribution point attribute ("supportsFileDebug") or by activating the debug extension together with the language extension and using the proposed provideFileDebugConfiguration.

Extension Activation:
"what would be the example scenario where the extension is not activated": see above.
You said: "If the extension is not activated then 99% it can not handle the open file, since it would already activated on that file type." With "extension" you mean language extension, right? But language extension and debugger extension do not yet have a dependency. So activating the language extension does not activate the debugger extension.

Run without Debugging:
Yes, with a contribution point attribute we could more precisely show/hide the actions: e.g. no "Run without Debugging" if the extension does not support it. If the contribution point attribute
is missing we would fall back to the current imprecise behavior.

[isidorn]

I was thinking that most extensions are actually a debugger and a language service in one (but that does not have to hold). In order to better decide on the activation strategy and if it needs a solution I would be interested in how many debuggers are out there that do not activate on opening of a file type they are interested in.
And I do agree they could activate on a new event tied with provideFileDebugConfiguration.

Run without debugging: makes sense.

[weinand]

Most prominently none of the node.js/JS debuggers are tied to the Typescript and Javascript language extensions...

[thernstig]

Apologize if this does not belong here, if so you can mark it as off-topic and I could potentially create a new issue.

It sounds as if this issue partially implements functionality of https://marketplace.visualstudio.com/items?itemName=formulahendry.code-runner. If this is true and that this issue implements run one file, then it would be cool to extend it to be able to run with the selected text i.e.

"Run selected code snippet in Text Editor"

[akaroml]

@testforstephen this is very interesting. I think Java debugger already supports debugging single files. Could you join the discussion here?

[weinand]

@thernstig this issue is more focused on debugging a single file than just running. So we want to provide another universal entry point into debug extensions. Any debug extension that is interested in improving its single file debug experience would be able to participate.

The extension you are mentioning above doesn't deal with debugging at all and it is not open for new languages.

[testforstephen]

Play Button:
I like the play button for single file debugging. Regarding the question of whether to enable the play button for the active editor, I tend to let the debugger tell the vscode in some sort of static way, such as a static contribution point attribute ("supportsFileDebug") as you mention. The proposed provideFileDebugConfiguration API should only be used after the play button is clicked. Trigger the debugging if the API returns a debug configuration (or an descriptive error if debugging isn't possible).

If you judge whether or not to enable play button by calling the proposed provideFileDebugConfiguration API, the cost can be a bit high. For example, a single Java file can be run only if it contains static main method. If you want to be accurate, you need to keep asking the debugger for a confirm when the user is editing the file. This is a bit over-designed. Static declaration of whether support for single file debugging is sufficient imho.

Keep single file config in memory or persist?
Also need consider whether to persist the in-memory config into launch.json. Maybe that's something for debuggers to consider. In current Java implementation, if no launch.json exists, the auto-generated single file config will be kept in memory. But if launch.json exists, then persist it for reuse and customization.

Run without Debugging:
Lastly considering Run without Debugging, it's more popular than debugging. In Java single file debugging support, we add Run and Debug CodeLens on top of a main method. And the click ratio between Run and Debug is about 7.5 : 1.

[weinand]

@testforstephen great feedback, thanks a lot.
We will consider this in May when we finish the work on this.

[gjsjohnmurray]

Using the latest vscode-mock-debug in a VS Code - OSS that I just built from master, the Run button makes the debug toolbar appear, then change, then disappear when the run has ended.

Can anything be done to prevent this unnecessary activity>

[int19h]

The toolbar is useful for long-running code even without breakpoints in the picture, since it provides the means to pause or stop the process. And a real adapter wouldn't know in advance if the code is long-running or not.

[isidorn]

@weinand yes I can introdcue the run group.
The context menu would not work in this case imho, since the two clicks is a bit cumbersome.

As for the $(play) and $(debug-alt-small) we simply need to resize them so they are the same size and aligned. Looking at them the problem is the debug-alt-small which is not designed to be used in a horisontal action bar. We should resize it, and if that is not possible (activity bar might get broken) then we simply need to introduce a new codicon. @misolori is it possible that you look into this?

@luabud @int19h thanks a lot for great feedback!
@gjsjohnmurray as @int19h already mentioned we do not know in advance how long the session will last, and thus we show the toolbar.

---

After compiling all the feedback and great ideas I suggest we start with @weinand initial proposal.
Two actions in the editor title area as the Mock Debug does today. If we get feedback that it produces clutter or that users want to customise this further we can change our recommended design.

Action items:

• Introduce run group @isidorn
• Introduce $(debug-alt-small) codicon which has a good size for the editor title area @misolori
• Communicate to all debug extensions what is our recommendation @weinand @isidorn
• Write a test plan item @isidorn @weinand

Let me know if this makes sense.

[gjsjohnmurray]

@int19h wrote:

The toolbar is useful for long-running code even without breakpoints in the picture, since it provides the means to pause or stop the process. And a real adapter wouldn't know in advance if the code is long-running or not.

but previously wrote:

To clarify, we do support "noDebug" in debugpy, and we specifically do this by not touching the debuggee at all - in that mode, launch.json controls what gets spawned and how, but that's that.

When in long-running Python code launched this way, what does the debug toolbar's 'pause' button do? Suspend execution, show the current line, and allow setting of breakpoints? Or merely pause the process until I press either 'play', 'stop', or 'restart'? Or something different?

[weinand]

@isidorn I will communicate the guidelines in the release notes.
I don't think that there is a need for a test plan item since the only new functionality is the "run" group which I will verify together with the improved icons with Mock debug. So I suggest we add the "verification-needed" label.

[weinand]

@gjsjohnmurray the debug toolbar's behavior is controlled by the debug extension/debug adapter.

Possible strategies:

• the debug adapter exits as soon as the program is launched. In this case the debug toolbar will only appear for a brief moment. @isidorn for this case it would be great if the debug toolbar would only show up after a short delay.
• If the debug adapter waits until the program exits, the debug toolbar will be there while the program is running. Typically the debug adapter will only implement the "stop" functionality, so the program, can be terminated by pressing the "Stop" button.

[gjsjohnmurray]

@weinand thanks for your input. It looks like Mock implements the second strategy. Perhaps add a compile time flag (cf its existing 'runMode' flag) to give us an example of a DA that implements the first.

And to help with understanding/testing of the differences, how about making its fireEventsForLine method interpret 'wait(2000)' as an instruction to simulate a long-running command (in this case 2000ms)? Doing this before the noDebug test will mean we can see how the debug toolbar's controls behave even when launching from the Run button.

[int19h]

@gjsjohnmurray For Python, it would do nothing - in fact, I believe we respond to that message with an error response. But there are many other debug adapters :) Besides, there's still Stop, which is always applicable.

@weinand Speaking of which - perhaps Pause should be surfaced as a capability, so that it can be disabled/hidden entirely, rather than relying on error responses for this? I suspect that most debug adapters wouldn't be able provide it in "noDebug" mode, so it should be a fairly common case.

+1 on only showing the toolbar after a short delay. This would cover the common case of a simple script nicely without regressing UX elsewhere; our users should like that.

[isidorn]

@weinand makes sense for the verifcation needed and release notes

@gjsjohnmurray @int19h can you please create a new issue to show the toolbar after a short delay, since this is not directly related to the single file debug discussion. Thanks!

[gjsjohnmurray]

@gjsjohnmurray @int19h can you please create a new issue to show the toolbar after a short delay, since this is not directly related to the single file debug discussion. Thanks!

#101018

[isidorn]

I have introduced the run group to the Editor Title area as we agreed upon, the run group is left of the navigation group.
I have also adopted this in Mock Debug via this commit. I have used the order 10 and 20 to give the group more flexibility if other extensions want to introduce actions between these.

I will update the docs here https://code.visualstudio.com/api/references/contribution-points#Sorting-of-groups
The open question is should the group be called 0_run which is a bit ugly. Will check with Ben.

I have created this follow up item for @misolori #101026

@weinand and me can work on finalising on what is the exact recommnedation for extensions and post it here and in the release notes.

[isidorn]

After discussion we decided to go with the group name 1_run to be better aligned with the current group names.
I have updated our docs to reflect the new group name.

---

To recap @weinand and me came up with the following recommendations for debug extensions (we will also cover this in our release notes):

1. Use the $(play) icon for Run, $(debug-alt-small) for debug
2. Put actions in the 1_run group, use order 10 and 20.
3. Use title "Run File" and "Debug File" or "Run Python File" and "Debug Python File"
4. Put either one or two actions based on your use case
5. Try to not to put other debug actions, if those are needed please let us know so we think about having a consistent experience

Mock debug example can be found here

---

fyi @connor4312 since we could also do this in js-debug

To verify: clone the mock debug repo from here, run the extension and make sure the commands are properly contributed for every markdown file in the editor title area.

[weinand]

Verified that the proposal works and looks fine.
The icons are not yet aligned, but I commented here: #101026 (comment)

[akaroml]

After discussion we decided to go with the group name 1_run to be better aligned with the current group names.
I have updated our docs to reflect the new group name.

To recap @weinand and me came up with the following recommendations for debug extensions (we will also cover this in our release notes):

1. Use the $(play) icon for Run, $(debug-alt-small) for debug
2. Put actions in the 1_run group, use order 10 and 20.
3. Use title "Run File" and "Debug File" or "Run Python File" and "Debug Python File"
4. Put either one or two actions based on your use case
5. Try to not to put other debug actions, if those are needed please let us know so we think about having a consistent experience

Mock debug example can be found here

fyi @connor4312 since we could also do this in js-debug

To verify: clone the mock debug repo from here, run the extension and make sure the commands are properly contributed for every markdown file in the editor title area.

@testforstephen please take a look at the recommendations and see how Java debugger can benefit.

[testforstephen]

@akaroml sure, tracked with issue microsoft/vscode-java-debug#834

[DanTup]

@isidorn

As for the $(play) and $(debug-alt) we simply need to resize them so they are the same size and aligned. Looking at them the problem is the debug-alt which is not designed to be used in a horisontal action bar. We should resize it, and if that is not possible (activity bar might get broken) then we simply need to introduce a new codicon. @misolori is it possible that you look into this?

Is this being tracked anywhere? It definitely looks a bit weird right now with different sizes.

[isidorn]

@DanTup yes, here #101026

[DanTup]

@isidorn aha, perfect - already shipped! :) I copied the original name (from #92269 (comment)) so if that's the main source for this info, might be worth editing in there (I did check the release notes, but it didn't seem to include the specifics).

Thanks!

[isidorn]

Makes sense, I have updated my comments from above. Thanks for pointing this out.