terminal: navigation mode more commands

[jvesouza]

Issue Type: Feature Request

Currently the terminal: navigation mode implements commands so that we can see lines of the content displayed in the terminal using a screen reader.
In my view it would be quite interesting if we could revisit this content by characters or words as well.
CC @webczat @isidorn

VS Code version: Code - Insiders 1.45.0-insider (44c5185, 2020-04-10T18:15:17.029Z)
OS version: Linux x64 5.6.3-arch1-1

[isidorn]

I think this feature request makes a lot of sense. Forwarding to @Tyriar so he can weigh in on this

[Tyriar]

@jvesouza the intent was that we expose line navigation to navigate the virtual list, then you can use your screen readers native capability to read through the line character-by-character. Does this not work?

[jvesouza]

@jvesouza the intent was that we expose line navigation to navigate the virtual list, then you can use your screen readers native capability to read through the line character-by-character. Does this not work?

It doesn't work consistently, although I believe the problem is with the orca.
When I use orca's flat review mode to navigate through characters, the contents of the line are read twice.
The same happens when I use the orca command 'speaks the current flat review line'.
CC @joanmarie

[Tyriar]

Sounds like this might be a bug then 🤔

[webczat]

I believe the correct way/intention? is that if you are on a output line you should go back to browse mode. so we are back to problems with application role? also, if I just do things like ctrl+up then esc, I return to input field. But, if I do ctrl+up, switch to browse mode, navigate even left/right, esc does not work even if I subsequently return to focus mode.

[Tyriar]

@webczat ok I see the problem, so if we stay out of browser mode how would you expect this to work? Also do we do something this anywhere else in the application?

[joanmarie]

As I stated in #94708, Orca on rare occasions IS getting the right / expected caret moved events (i.e. like it gets from a native terminal) when you left/right arrow. And when it gets those, Orca presents them just like it does in a native terminal. If we can figure out why those events are often missing, I think this bug would be largely solved, would it not?

@webczat : As an aside, Orca never captures/steals escape. Thus the mode shouldn't matter.

@jvesouza: I'll look into the double presentation problem when using flat review. Thanks!

[webczat]

@joanmarie as I said, even returning to focus mode after movement doesn't help. So maybe orca does not steal esc, but something that happens either when entering browse mode or moving makes esc not work.
@Tyriar I would expect that to switch to browse mode. Othervise I would be surprised if it just worked unless you would add some special support for arrow keys.

[webczat]

Actually I believe this case is not the only one where esc may stop working. sometimes you have to tab to relevant field to use it, I am not sure if you lose focus or what...

[webczat]

Did some testing. Just entering browse mode does not break esc, but browse mode allows me to navigate, and if I move (even one character left or one line up while in browse mode) esc stops working. Tabbing twice is required to get back to input field.

[joanmarie]

When you are in browse mode and using caret navigation, Orca sets the caret at the new location. It does this for a variety of reasons including:

1. removing focus from items which should no longer have focus
2. having the caret in the right place for (browser-driven) text selection

I don't know if the caret being moved is causing VSCode's escape to cease functioning. But if Orca didn't set the caret in browse mode, other stuff would break.

[webczat]

Hmm, but if I am still moving inside of the terminal output and didn't leave it, shouldn't I essentially stay on it?

[joanmarie]

Hmm, but if I am still moving inside of the terminal output and didn't leave it, shouldn't I essentially stay on it?

That depends on what you mean by "on" and "it". Executive summary, you wind up being on a descendant of the terminal; you don't stay directly on the element serving as the entire terminal.

Looking at the terminal with the inspector, I see that it is a div with a div with a list. The list is made up of multiple ARIA role listitem instances, one item per line. So when you switch to Orca's browse mode and use Orca's caret navigation, Orca sees list items with text. List items with text are things Orca will set the caret in for the reasons I described earlier.

Without using Orca, but just using Accerciser, if I select one of the list items in Accerciser's tree of accessible objects, switch to Accerciser's iPython console, and type:

acc.queryText().setCaretOffset(0)

I see a focus rectangle drawn around the list item.

Whether or not this is causing a state change in VSCode that causes Escape to stop working, I couldn't tell you. @isidorn and/or @Tyriar: Thoughts?

[joanmarie]

When I use orca's flat review mode to navigate through characters, the contents of the line are read twice.
The same happens when I use the orca command 'speaks the current flat review line'.

@jvesouza I believe I have this particular issue fixed in Orca master. Mind trying? Thanks!

[isidorn]

@Tyriar would know better if he is eating up the Escape key in the terminal.

[Tyriar]

Escape exits the navigation mode:

{ "key": "escape",                "command": "workbench.action.terminal.navigationModeExit",
                                     "when": "accessibilityModeEnabled && terminalA11yTreeFocus" },
{ "key": "ctrl+down",             "command": "workbench.action.terminal.navigationModeFocusNext",
                                     "when": "accessibilityModeEnabled && terminalA11yTreeFocus || accessibilityModeEnabled && terminalFocus" },
{ "key": "ctrl+up",               "command": "workbench.action.terminal.navigationModeFocusPrevious",
                                     "when": "accessibilityModeEnabled && terminalA11yTreeFocus || accessibilityModeEnabled && terminalFocus" },

When navigation mode is not active, escape is eaten by the terminal as it's sent to the shell (eg. it's needed in vim).

[webczat]

the problem was just that if I move the orca caret just a bit but still inside of terminal output the esc stops working. I am not on the input at this time

[Tyriar]

/duplicate #69795