Prominently recommend language extensions for debugging

[digitarald]

Problem

If a user misses to install an important language extension that provides debugging (like python or java), the setup of run & debug makes a lot less sense. The provided defaults in "Select Environments" are all meant for JS and don't give any additional context.

In usertesting we saw most Java/Python users without the right extension give up on "run & debug", and one even try Chrome-debugging on a python file. None of them clicked on the nondescript "More …" in "Select Environment" (which opens the search for debugger extensions).

Solutions

The run & debug viewlet and setup prompt could use file and workspace recommendations to highlight a shortlist of extensions that the user should install to make debugging easier.

We could start with a small scope for Java, C++ and Python and run this as experiment.

cc @sana-ajani @stevencl @isidorn

[isidorn]

@digitarald what you are saying makes sense. I can look into improving this flow next milestone, thus assigning to April.
For starters I believe the More... should be the first option when there is no matching between the active language and the installed debuggers.
To nicely recommend a Python extension for example that would have to be hard coded and a bit of a hack (similar to how our extensions recommendations work today).

fyi @weinand

[isidorn]

I have improved this quick pick dialog.
If a user does not have python installed and starts debugging previously we would show this dialog

Now we show the following dialog

Clicking on the Install item shows the extension viewlet with the correct "python" query

I think this really improves the flow and am quite happy with how this turned out, and for now I would not put any more prominent recommendation measure.
Let's see how users behave to this and then I would be open for additional changes.
There is nothing hard coded here, we are just looking at the language of the active editor, so this should work for all debuggers.
Ideas for further improvement are welcome, especially wording

fyi @connor4312 @weinand

[sana-ajani]

Question, when does this dialog below show versus the quick pick of environments then? I don't have C++ extension, but I created a .cpp file and started debugging, and I see this dialog below. On the other hand, when I don't have the Java extension, but I create a .java file and start debugging, I see the quick pick.

Small nit for the wording for the quick pick box: "Cannot find extension to debug Python" and the option should be "Install Python extension..."

[isidorn]

@sana-ajani good point. I have changed to use the languageLabel not the languageId, so now it will be Python.

C++ works just fine for me, can you please clarify your steps.
I believe you have some extension which is affecting this flow. Like C++ or LiveShare... Also that error message is not coming from VS Code core.

[connor4312]

I think Install an extension for C++ would work better for wording

[isidorn]

Yeah I like that, doing the change... Thanks!
Updated text

[digitarald]

This looks great! I will spend some more time tomorrow to test the flow.

A wording aligned with the extension recommendation prompt could be "Show recommended debugger extensions for C++", being clear about showing (vs install) and the value that this will allow debugging.

I am not sure about how this potentially clashes with the placeholder text. "can not find extension …" vs "show extension …" (why show what can't be found?) could maybe need to be a more crisp. Maybe something along the lines of "Debugging C++ requires an additional extension".

[isidorn]

"Debugging C++ requires an additional extension" works for me as a placeholder text, I can make that change.
However "Show recommended debugger extensions for C++" I dislike, because they are not recommended, we are not using the same mechanism as for the recommendations so I would not want the user to confuse these.
Using the word "Show" instead of "Install" works for me.

Let's also see if @weinand has feedback and I might demo this in Zurich standup

Thanks a lot for sharing your thoughts!

[digitarald]

However "Show recommended debugger extensions for C++" I dislike, because they are not recommended, we are not using the same mechanism as for the recommendations so I would not want the user to confuse these.

If it doesn't use the same logic, the "recommended" should be dropped – right. As a consideration – could this use the same mechanism if the language has a recommended important extension?

[weinand]

@isidorn by just reading the comments from above, I do not really understand what a user is expected to type into the Quick Pick.
If nothing is expected from the user then I suggest that we use a modal dialog with an explaining text and some buttons.

[isidorn]

@weinand that's a good point. I have switched to use a modal dialog
Here's how the first version looks

[isidorn]

The button will not start the installation process automatically it will just open the extensions viewlet, so maybe the more correct term would be as @digitarald suggests: "Show Python extension"

[weinand]

@isidorn yes, using a modal dialog in this situation is more intuitive.

Since the next step after this dialog is to "find" a Python extension on the Marketplace, I suggest to not use the word "find" already for the first step.

We could use something like this:

You don't have an extension for debugging Python.
Should we help finding Python extensions on the Marketplace?

And for the Button:

Find Python extensions

[isidorn]

@weinand thanks for feedback. Here's Version 2.
Native speaker feedback welcome :) @connor4312

[isidorn]

Verifier:

1. Open a Python / C++ file and do not have an Python / C++ extension installed. Press F5, make sure you get the above dialog properly guiding you to install an extesnsion
2. Once you have the extension installed you do not get this dialog

[digitarald]

@isidorn any chance that we can have this behind an experiment flag to observe improvements in debug sessions. Debugging is a new-user issue across all language extensions; so those insights could help to further invest into this setup flow. I'd be happy to help set everything up and shepherd it through the process.

[isidorn]

@digitarald thanks for the offer but I think experiments make good sense when we are not sure about what path to take.
Since I am 100% sure this is an improvement to what we had previously I would prefer if we did not do an experiment and instead listen to user feedback directly.

[digitarald]

Since I am 100% sure this is an improvement to what we had previously I would prefer if we did not do an experiment and instead listen to user feedback directly.
I think experiments make good sense when we are not sure about what path to take.

I agree that we have high confidence for this one and don't need to evaluate options – an experiment isn't blocking. We'd like to get better at understanding the cycle of qualitative observations and quantitative impact; with the help of running these smaller high-confidence improvements through experiments. We would better understand how improvements change metrics to further improve metrics and baseline future experiments. This is especially important for debugging, being one of VS Code's core features but also one of the steepest adoption curves.

Some of the questions that we could observe beyond an increased debugger setup rate:

• Which languages does this help most? Is this best for new users, as we currently think, or even help advanced users to set up debugging.
• Are there unintended side effects for this prompt that need further tweaking, like the search triggering for markdown files?
• Do more users successfully debug – do they just run single files, whole project, use breakpoints?
• If the impact is high, it can help inform to invest in further streamlining debugging. If not, revisit how prompt participants in usertesting (cc @stevencl).

[sana-ajani]

@isidorn @digitarald I think it would be valuable to see just how much this prompt helps new users get the right extensions installed and how exactly it helps the run and debug flow.

Running as an A/B test in Insiders before we release would be a good confirmation, or I might suggest doing a few user studies since this is a problem we initially saw from user testing (though this might take more time to set up and parse through results)

[connor4312]

Search is improved when switching to @category instead of tags:

[isidorn]

@connor4312 thanks!

[digitarald]

Would it still further improve the results by removing the @sort:installs, @isidorn?

[isidorn]

@digitarald yes, I have this on the plan for today, here's the item tracking this #122079
Thanks

[digitarald]

Ha, lost track with search improvements landing across different issues. Thank you, @isidorn!