Injected text accessibility

[isidorn]

Testing #129383

Here's how I think the injected text accessibility should work.
Reading the injected text every time it shows up would be very aggressive and spammy. So I suggest to have an explicit command which the users could trigger that would help them here. Here's how the command would behave:

• once the user triggers the command the injected text is read out
• focus is moved to the injected text hover so the user can navigate / accept a suggestion
• once the user accepted or closed the hover, focus is back in the editor
• command would have some keybinding. Something with a key chord should be empty

I see that Cmd + K, Cmd + I already triggers the hover. However the problem here is:

• focus is not moved into the hover
• the suggested insertion is not being read out

More feedback can be found here #125294

[hediet]

Thanks for testing it!

focus is not moved into the hover

Hmm, are you sure?
Ctrl K, Ctrl seems to move the focus to the hover for me on windows (focus is indicated by the blue border):

[isidorn]

@hediet I mean for your copilot hover, not in general. Previous and next buttons are disabled in my case, maybe that is the cause of why the focus is not moved?

[isidorn]

@hediet just to clarify my idea, since you mentioned in slack
"by showing the inline suggestion in the inline suggestion hover when the screen reader mode is enabled"

I think a nicer implementation would be to always have the aria-label set, even if the screen reader is not there. Since the aria label will only be visible to screen reader users. So I would not change the actual content of the widget.

Though what you mentioned will also work, and feel free to try that approach out first.

[hediet]

This is my idea:
When this.accessibilityService.isScreenReaderOptimized() returns true, the inline suggestion is added to the hover:

• Screen reader users are informed that an inline suggestion follows when the hover is triggered
• When clicking on next, the hover is updated & the screenreader automatically reads the inline suggestions
• This does not affect non-screenreader users
• The text fixes the focus problem

What do you think?

[isidorn]

I think this is a lovely solution. I really like it. Though I would still have to try it out.
The only remaining piece of the puzzle is: how to improve that the screen reader users are informed that there is a hover at all. But we can tackle that afterwards.

However I checked your PR and there are a lot of code changes for last day of endgame, and since we never supported this I suggest that we simply do this first week of debt. That way you can merge it in insiders and I can test it out.
Due to that I am moving this to August. Hope that makes sense, thanks!

[hediet]

Thanks! Makes a lot of sense!

[hediet]

Reopening, as it is unclear if the PR fully addresses this issue.

[isidorn]

@hediet I can give this a test spin. Are you happy with the current state of things so I try it out?

[hediet]

Yes, you can try it!

[isidorn]

Oh the insiders is not yet out with your changes since we freeze insiders this week.
I will give this a spin once we release in a couple of days.

[hediet]

This is the outcome:

When screenreader mode is enabled and the cursor is before an inline suggestion, hitting Ctrl + K, Ctrl + I will open a hover and the screenreader will read the suggestion.
If "Tab moves focus" is set, "Tab" will then cycle through various commands regarding inline suggestions (next/previous/accept/...). "Enter" will invoke the currently selected command.

I think this addresses all points mentioned in this issue.

[zersiax]

All, but one, which was already brought up by @isidorn . We are going to have to come up with a way to indicate the hover is there to begin with. It's great that the hover is accessible, but it's not going to do much good if people don't know it should be invoked, given this is a manual process.
@isidorn was any thought given yet to other line-based suggestions like lightbulbs, error indicators etc. yet in other issues? It feels to me like the best solution here would be to tackle all these at once and give CoPilot that framework to just hook into?

[hediet]

I opened #130565 to track how to notify screenreader users when there are inline suggestions available.