Simplify keyboard navigation through blocks

[afercia]

Follow up to #2031. The accessibility team has identified a list of 10 high priority issues for version 1 and agreed with the Gutenberg team to mark them with the "merge proposal" milestone.

For keyboard and screen reader users, keyboard navigation through the blocks needs to be greatly simplified. Feedback received from accessibility testers so far has always reported that navigation through the UI is confusing and terribly complicated: there are just too many tab stops, too many controls, formatting toolbars, block movers, ellipsis buttons, and the like. As long as the amount of blocks in a post increases, the UI controls increase exponentially, adding several tab stops for each block.

This issue was discussed at length in #2031 and one of the identified solutions was trying to implement two keyboard interaction modes: "navigation mode", and "edit mode". There's an initial PR that started experimenting with this, see #3195.

Trying to summarize the basic idea;

• it is meant for keyboard interaction, shouldn't affect pointing device users
• say in a post there are 20 blocks
• the initial state of the UI for keyboard users should be "navigation mode"
• in "navigation mode", tabbing through the 20 blocks moves focus through each block: this way, in our example, with 20 tab presses users would be able to navigate through the 20 blocks
• once a block has focus, pressing Enter switches the block to "Edit mode"
• in "Edit mode" users can normally operate on the edited block
• pressing Escape switches the UI back to "navigation mode"
• users can continue their quick navigation through blocks

The accessibility team agreed this could be a very interesting idea (VIM users might like it ;) ) and probably one of the few options to guarantee a decent level of keyboard accessibility at this point of the project. Open to other suggestions, but we'd like to strongly recommend the interaction model described above.

Of course, once implemented, this should be clearly documented and made available in a User Guide, see #5420.

[rianrietveld]

I highly endorse this approach in combination with a solid user guide.
The way the keyboard navigation is now, is too complicated and confusing.

[nic-bertino]

Some notes from the a11y team Slack:

I think the UI complexity is universal to both pointed device users AND keyboard users. This is a challenge with the fundamental concept of blocks (or, at least, how they articulate themselves in Gutenberg).

Consider this: if you have 20 paragraph blocks, that means you are reproducing the same UI 20 times. 20 consecutive paragraph blocks, though, should be treated as one cohesive piece of content. This is a common pattern between document editors, Medium, virtually everything.

If that’s extended to other basic content-like items (headings, lists, and similar non-blocking elements), I think it reduces the amount of cognitive load while reducing the amount of meta information that needs to be relayed to an assistive device. Instead, it’s relaying the content itself, not the interface.

I think blocks become more important as the layouts become more complex, but perhaps we can lessen the load for the user by treating consecutive pieces of text-based content as single blocks. They can still register in React as individual blocks, but at the very least, they should be presented in the editor as a single cohesive block.

Also, because this is a common approach to this type of content, I think we have some information, patterns, and expectations on how to present this UI to assistive technologies, which should help inform how to address this moving forward.

Here's a quick mockup I did to explain the above.

[abrightclearweb]

Blocks - keyboard interaction (Tab, Shift+Tab)

/* Updated for Gutenberg 2.7 */

Paragraph block

Tab:

Initial focus: cursor in to add text

1. Add image
2. Add table
3. Add heading
4. Add block button
5. Next block

Shift + Tab:

1. Add block button (horizontal line above)
2. Focus on whole block
3. Previous block

Paragraph block, content added

Initial focus: on paragraph block

Tab:

1. Add block button (horizontal line above)
2. Move up
3. Move down
4. More Options
5. Remove
6. Cursor in block
7. Next block - or Document panel if it’s the last block.

Shift + Tab:

1. Previous block

Paragraph block, content added

Initial focus: in text area, cursor at end

Tab:

1. Next block

Shift + Tab:

1. Remove
2. More Options
3. Move down
4. Move up
5. Add block button (horizontal line above)
6. Focus Paragraph block
7. Previous block

Image Block

Initial focus: on image block

Tab:

1. Add block button (horizontal line above)
2. Move up
3. Move down
4. More Options
5. Remove
6. Upload
7. Media Library
8. Next block

Shift + Tab:

1. Previous block

Image Block, image just added

Initial focus: on image

Tab:

1. Caption
2. Next block

Shift + Tab:

1. Remove
2. More Options
3. Move down
4. Move up
5. Add block button (horizontal line above)
6. Focus Image block
7. Previous block

Image Block, image added

Initial focus: on image block

Tab:

1. Add block button (horizontal line above)
2. Move up
3. Move down
4. More Options
5. Remove
6. Caption
7. Next block

Shift+Tab:

1. Previous block

Image Block, image added

Initial focus: on caption field

Tab:

1. Next block

Shift + Tab:

1. Link
2. Strikethrough
3. Italic
4. Bold
5. Remove
6. More Options
7. Move down
8. Move up
9. Add block button (horizontal line above)
10. Focus on Image block
11. Previous block

Heading Block

Initial focus: cursor in to add heading

Tab:

1. Next block (Write heading… appears in the block)

Shift + Tab:

1. Remove
2. More Options
3. Move down
4. Move up
5. Add block button (horizontal line above)
6. Focus on Heading block
7. Previous block

Heading block with Heading added

Initial focus: on Heading block

Tab:

1. Add block button (horizontal line above)
2. Move up
3. Move down
4. More Options
5. Remove
6. Focus on text area, cursor at start
7. Next block

Shift + Tab:

1. Previous block

Heading block with Heading added

Initial focus: cursor in Heading text area

Tab:

1. Next block

Shift+Tab:

1. Remove
2. More Options
3. Move down
4. Move up
5. Add block button (horizontal line above)
6. Focus on Heading block
7. Previous block

Gallery Block

Initial focus: on gallery block

Tab:

1. Add block button (horizontal line above)
2. Move up
3. Move down
4. More Options
5. Remove
6. Upload
7. Add from Media Library
8. Next block

Shift + Tab:

1. Previous block

Gallery Block, after images added

Initial focus: unclear

Tab:

1. Upload an image
2. Next block

Shift + Tab:

1. Remove
2. More Options
3. Move down
4. Move up
5. Add block button (horizontal line above)
6. Focus on gallery block
7. Previous block

Can’t tab to Write caption, Remove image (available on mouse click)

Gallery Block, after images added

Initial focus: on gallery block

Tab:

1. Add block button (horizontal line above)
2. Move up
3. Move down
4. More Options
5. Remove
6. Upload an image

Shift + Tab:

1. Previous block

List block

Initial focus: cursor in unordered list, ready for item 1

Tab:

1. Next block (Write list… appears in the block)

Shift + Tab:

1. Remove
2. More Options
3. Move down
4. Move up
5. Add block button (horizontal line above)
6. Focus on list block
7. Previous block

List: after items added

Initial focus: on list block

Tab:

1. Add block button (horizontal line above)
2. Move up
3. Move down
4. More Options
5. Remove
6. Cursor at start of list item 1
7. Next block

Shift + Tab:

1. Previous block

Quote

Initial focus: cursor in Write quote box

Tab:

1. Write citation
2. Next block

Shift + Tab:

1. Remove
2. More Options
3. Move down
4. Move up
5. Add block button (horizontal line above)
6. Focus on Quote block
7. Previous block

Quote, content added

Initial focus: on quote block

Tab:

1. Add block button (horizontal line above)
2. Move up
3. Move down
4. More Options
5. Remove
6. Cursor in start of quote field
7. Cursor in start of citation field
8. Next block

Shift + Tab:

1. Previous block

Audio block

Initial focus: Enter URL of audio file here…

Tab:

1. Use URL button
2. Upload button
3. Media Library button
4. Next block

Initial focus: Enter URL of audio file here…

Shift + Tab::

1. Remove
2. More Options
3. Move down
4. Move up
5. Add block button (horizontal line above)
6. Focus Audio block
7. Previous block

Audio block, content added

Initial focus: unclear

Tab:

1. Focus on audio player
2. Play/Pause
3. Playthrough slider
4. Mute/Sound
5. Volume slider
6. Download
7. Bold
8. Italic
9. Strikethrough
10. Link
11. Name?
12. Next block

Shift + Tab:

1. Remove
2. More Options
3. Move down
4. Move up
5. Add block button (horizontal line above)
6. Focus on audio block
7. Previous block

Audio block, content added

Initial focus:: whole block

Tab:

1. Add block button (horizontal line above)
2. Move up
3. Move down
4. More Options
5. Remove
6. Focus on audio player
7. Play/Pause
8. Playthrough slider
9. Mute/Sound
10. Volume slider
11. Download
12. Bold
13. Italic
14. Strikethrough
15. Link
16. Name?
17. Next block

Shift+Tab:

1. Previous block

Cover Image

Initial focus: on whole block

Tab:

1. Add block button (horizontal line above)
2. Move up
3. Move down
4. More Options
5. Remove
6. Upload button
7. Media Library button

Shift+Tab:

1. Previous block

Cover image, with content added

Initial focus: on image

Tab:

1. Bold
2. Italic
3. Strikethrough
4. Link
5. Add text
6. Next block

Shift + Tab:

1. Remove
2. More Options
3. Move down
4. Move up
5. Add block button (horizontal line above)
6. Focus on cover image block
7. Previous block

Cover image, with content added

Initial focus: on block

Tab:

1. Add block button (horizontal line above)
2. Move up
3. Move down
4. More Options
5. Remove
6. Bold
7. Italic
8. Strikethrough
9. Link
10. Enter text
11. Next block

Shift + Tab:

1. Previous block

Subhead

Initial focus: in text box

Tab:

1. Next block (Write subhead… appears in box)

Shift + Tab:

1. Remove
2. More Options
3. Move down
4. Move up
5. Add block button (horizontal line above)
6. Focus on subhead block
7. Previous block

Subhead, with content

Initial focus: text box, at end of text

Tab:

1. Next block

Shift + Tab:

1. Remove
2. More Options
3. Move down
4. Move up
5. Add block button (horizontal line above)
6. Focus on subhead block
7. Previous block

Video block

YouTube URLs don’t seem to work!! (YouTube is a separate block)

Initial focus: Enter URL of video file here…

Tab:

1. Use URL button
2. Upload button
3. Media Library button
4. Next block

Shift + Tab:

1. Remove
2. More Options
3. Move down
4. Move up
5. Add block button (horizontal line above)
6. Focus on video block
7. Previous block

Video block, content added

Initial focus: unclear

Tab:

1. Focus on video
2. Play/Pause
3. Scrub through timeline
4. Mute/Audio
5. Volume slider
6. Full screen
7. Download button
8. Bold
9. Italic
10. Strikethrough
11. Link
12. Add title?
13. Next block

Initial focus: whole block

Shift + Tab:

1. Previous block

[rianrietveld]

@abrightclearweb thanks Claire!

[samikeijonen]

Thanks @abrightclearweb! Can you write short summary what was good and bad about those keyboard interactions?

Are we still after the original "navigation" and "edit" mode as mention in the first comment?

Here is my short summary of keyboard interaction from block to block and inside a block:

• At the moment Esc doesn't do anything when editing block.
• Using up and down arrow keys user can scroll between the blocks. Naturally it stops in every line in paragraph, list etc.
• There are lot's of Tab stops inside a block but user can access the toolbar and settings.

[abrightclearweb]

Some of the annoyances about keyboard interactions:

1. Focus styles are not always clear enough e.g. when focus is on a whole block. I'm having to use the Focus Indicator Chrome extension to check where I am.

2. Image, Gallery, Cover Image: when you add these, I think focus should be set to the Upload button initially. You need to press Tab 6 times to get there. Could it go to Upload initially, then remove block etc.?

3. Tab order doesn't always make visual sense. e.g. if I'm in a Paragraph block and tab, then I tab through the last three blocks I used. So that might be Image, Gallery, Cover Image, then Add block. But on screen from left to right the order is: Add block (far left), Cover Image, Gallery, Image. (See screenshot.)

4. The horizontal line add block button is read out by VoiceOver as "Insert block, button". But how would a sighted keyboard user know what it does? All they see is the focus on a line. They might assume it adds a horizontal line.

5. If I tab out of the Add block panel, I end up focusing on the browser tabs. Shouldn't I go back to where I was?

6. There's a problem with the bold, italic, strikethrough and link controls overlapping the audio controls - see screenshot.

7. @nic-bertino makes a good point about not wanting many similar blocks to be discrete entities. But we know we should use proper headings and heading levels in our markup, and Heading is a separate block from Paragraph.

8. Is there a keyboard shortcut yet to get to the Block settings panel?

[afercia]

@abrightclearweb 👋 thanks so much for your testing and feedback. I'd absolutely agree the Tab order doesn't make much sense in some cases, some focus styles are too light (especially for my poor eyesight), etc. Some of these points have specific issues:

So that might be Image, Gallery, Cover Image, then Add block. But on screen from left to right the order is: Add block (far left), Cover Image, Gallery, Image.

This is fixed on master :)

The horizontal line add block button is read out by VoiceOver as "Insert block, button". But how would a sighted keyboard user know what it does?

Yeah, this was reported by many, see #6009 and hope to see some progress there /Cc @karamtosed please

If I tab out of the Add block panel, I end up focusing on the browser tabs. Shouldn't I go back to where I was?

😬Yeah... this is because, actually, the panel is at the bottom of the DOM (I've explained this to the dev team many times but they have technical reasons to keep it there). However, then it should be treated like a modal see #5242

Is there a keyboard shortcut yet to get to the Block settings panel?

Ctrl + backtick allows to cycle through the 3 main areas of the editor. What's needed here is a mechanism to customise the shortcuts, see #3218
At the end of the block settings sidebar, keyboard users will find a "Skip back to the selected block" link that appears only on focus.

@samikeijonen

Are we still after the original "navigation" and "edit" mode as mention in the first comment?

@youknowriad worked on a PR and implemented it, see #5709 but... there's a problem: keyboard events on a DIV element don't work with screen readers when in "browse mode", which defeats the purpose of that change. It works only in VoiceOver because VO doesn't use browse/forms mode. Any idea and testing regarding that PR would be greatly appreciated!

[afercia]

Re: the tab order, there's also #1934. I'd like to add an animated GIF to visually illustrate the order the UI controls in a block (a paragraph) are navigated when tabbing with the keyboard. Two considerations:

• does the order makes sense? Especially the "jump" from the "movers" to "More options/Remove" is a bit weird. As far as I see, some small adjustments are possible, to some extent. Also, would the proposal to add an option to place the block toolbar after the editable area be beneficial? See Block toolbars position counterintuitive for keyboard users #3976 (comment)
• what if the block toolbar had only one tab stop and navigating through its controls would be possible with the arrow keys? See Unexpected reverse-tab behaviour #1934 (comment)

[rianrietveld]

Feedback from @conlaccento on make/accessibility gutenberg testing:
Keyboard test on Gutenberg 2.7, on Windows 10 with Chrome v.66.
Some notes on my test:

• When I tab a new block is created, and if I tab again I have access to a menu with three icon on the right of the block (recently used?) and plus icon on the left to choose the block to add: it could be confusing to have icons surrounding the block and, most of all, tabbing I navigate the icons from right to left
• if I move between blocks with up and down arrows I can’t always see the focused block
• with the fix toolbar I was not able to access the bar with the focus on the block
• I was not able to access the block options on sidebar with the focus on the block

[rianrietveld]

@aardrian, can you have an opinion about @afercia's solution in
#5694 (comment)
Will this be understandable you think?

[aardrian]

@rianrietveld I am struggling because I see this is a long comment thread and I don't know where I need to start to get sufficient context.

That said, the first thing I did was look at the animated GIF (because it was moving) and my first reaction is that I would flag that tab order in an audit (1.3.2 Meaningful Sequence and 2.4.3 Focus Order). I don't know if they are failures, but my gut tells me to look under the hood because they may be.

Further, the icons not appearing until focus is a big usability issue in my opinion (and experience) because there is no cue to a sighted user that there are controls that may receive focus. When I think of a user with a screen magnifier when the focus leaves the magnified area, that user may not think to look right or left because that user saw no controls there previously. Training can mitigate this, but it is not a good first interaction.

Plus I have an issue with hiding controls in the interest of 'simplifying' user interfaces. That often does the opposite. It appeals to a design aesthetic, but in testing with users hidden controls can/do increase cognitive load and require interaction to identify all the options.

Now, to @afercia's bullets:

• does the order makes sense? […]

• If I understand the animation correctly, it starts in the content area, then goes up to some bar (which is partially obscured by the toolbar), to the left to arrows, to the right to icons, and then up to the toolbar. WCAG issues aside, I find it confusing but trainable.

• what if the block toolbar had only one tab stop and navigating through its controls would be possible with the arrow keys?

• It appears the toolbar conforms to the ARIA definition of one. I think it will be more technically correct (the best kind of correct, yeah?) but not as usable unless you also train users about the arrows. Certainly having it as a single tab-stop is preferable for skilled screen reader users.

[afercia]

Thanks @aardrian

I have an issue with hiding controls in the interest of 'simplifying' user interfaces. That often does the opposite. It appeals to a design aesthetic, but in testing with users hidden controls can/do increase cognitive load and require interaction to identify all the options.

FWIW I have the same issue 🙂

Re: the toolbar, we've discussed it during today's a11y meeting and agreed to try the ARIA interaction model. But then it should be A/B tested.

[afercia]

Out of curiosity, I've checked the order of the block controls in the "responsive" view, and seems to me it makes more sense (except the "horizontal line" a.k.a. "sibling inserter" is almost completely hidden by the top toolbar). A "mobile" toolbar appears after the block and groups the "Add block" button, the movers, ellipsis menu and remove:

Of course, the bottom toolbar could be an issue with very long blocks.

[afercia]

With regards to the experimental switch between "navigation" and "edit" mode tried in #5709 worth considering there are also alternate approaches to evaluate. See last comment on #5709 (comment)

[jasmussen]

Let's try and unstick this and make some decisions on how to move forward. Based on discussion in #5709, here are mockups for an approach I feel is clear, simple, unintrusive, and inspired by how notion.so does it, so you can test it there as a prototype.

Here's our content — for this exercise, imagine the Quote being a container for nested paragraphs. Note how no block is yet selected, but the editing canvas has focus:

Now press Tab, and you get this:

Press Tab again, to get this:

Press Tab once more, to get this:

Now press Enter, and you're now editing the first paragraph of the quote:

Now we press Escape, and we're back here:

Some notes:

• Enter enters editing mode, Tab takes you back to navigation mode
• The focus style in navigation mode is identical to the hover style, only there is no side UI to tab into when in navigation mode.
• When you are in Editing mode, Tab takes you through all the side UI and toolbar, then to the next block, just like it works in master today.

The benefit to the above is that to the user, this isn't really additional cognitive weight. It is visually almost indistinguishable from what we have today, but it accomplishes the goal of allowing you to press tab three times to get to the third block. In other words, if you never learn about navigation mode, it does not get in your way.

This approach is inspired by what notion.so does — only they use arrow keys instead of Tab to jump between blocks. But regardless, you can try notion to experiment with this approach as a prototype. Here's a GIF as well:

What are next steps? Can we make a decision to move forward with this?

[tofumatt]

What are next steps? Can we make a decision to move forward with this?

My greatest apologies in not seeing this sooner.

We should do (almost) exactly what you suggest, @jasmussen. I would be okay with keyboard arrows being a way we move through blocks in navigation mode but I suppose Tab/reverse-Tab is a fine start.

My one question is around Esc as an option over Tab for entering "Navigation" mode; it's more vim-like but it's also maybe a bit more expected? You are "leaving" one mode for another, and right now with editing focus Esc does nothing. I think it'd be a good UX.

[tofumatt]

(Optimistically adding to the milestone as a priority marker; not really a blocker but it's the nicest-nice-to-have I can think of.)

[afercia]

This approach was already tried in #5709 and doesn't work when a screen reader is running. I'd recommend to re-read what's in #5709 and familiarize with the concept of "browse mode / focus mode" used by screen readers on Windows before spending time on an implementation that would be not effective.

VoiceOver is really an exception to this model, I'd strongly recommend everyone to familiarize with screen readers on Windows. In short, in the default "browse mode", screen readers intercept keystrokes and pressing Enter on a div doesn't do anything. This was already discussed with @aduth, who also took some time to have a look at the NVDA source code to get an idea of how screen reader modes work 🙂and I guess he could provide some in-depth feedback if necessary.

As mentioned in #5709, there's the need of a button element to switch from a block "navigation mode" to "edit mode".

[mtias]

Consolidating with #4382

[joedolson]

In my opinion, this issue is probably the single biggest accessibility concern with Gutenberg right now. The sheer cognitive load of figuring out which direction you need to go to find a control at any given time and the number of controls required to navigate through the page transform a set of accessible micro-interactions into an unusable macro environment.

For any user of assistive technology, this is a massive usability challenge. For a sighted keyboard user, there's at least some visual affordance giving you navigational clues, but for non-sighted users, it's incredibly difficult. Solving this is a must-have, not a nice-to-have.

[tofumatt]

We're working on it right now in #10545. 😄

I'm making some accessibility suggestions based on screenreader testing. It should be posted within ten minutes.

[afercia]

We're working on it right now in #10545. 😄

Waiting to see and test what comes out of #10545, I would like to note that PR should have been labelled with the Accessibility label to make the a11y specialists in this community aware of it.

[youknowriad]

I would like to note that PR should have been labelled with the Accessibility label to make the a11y specialists in this community aware of it.

I would like to note that the PR has been labeled with "Need Accessibility Feedback"

[youknowriad]

I would also like to remind people to drop the "lesson giving" tone as a method of communication as it just creates frustraction and doesn't help things moving forward.

[afercia]

I would like to note that the PR has been labeled with "Need Accessibility Feedback"

Thanks, I've noticed it. The two labels have a quite different use, as "need feedback" gets removed (and it was removed) as soon as feedback has been given. Instead the general "Accessibility" label stays so contributors can still search based on that label. I'd say it's a pretty significant difference.

A proper label handling is important also for mutual professional respect. The accessibility team has kindly asked to be pinged for feedback, labels are an important part of that. I'm sorry if the tone sounder not proper, I was just in a rush because I was reviewing that PR.

On a side note, I've re-added also "Need Accessibility Feedback" as I think the previous feedback was incomplete (see comment on the PR) and likely there will be the need for more feedback with future iterations. Thank you.

[tofumatt]

I think this is addressed by #10545. 🙂 It allows navigation throughout the blocks, including with Firefox + NVDA in my testing. A keyboard shortcut is provided that allows easy access, even when the user is in a block that may present issues when trying to navigate out of using only the keyboard (a few of these, I think, do exist).

Please let me know if I'm off on this or there are bugs with #10545 in follow-up issues we can address.

[afercia]

A keyboard shortcut is provided that allows easy access

Worth noting with VoiceOver the Blocks navigation menu shortcut Control Option o doesn't work because Control Option is the default modifier used by VoiceOver, so the blocks navigation is not accessible to VoiceOver users. I've found other shortcuts affected, going to open a separate issue.

[afercia]

Discussed this issue during today's accessibility bug-scrub and the team agreed the “Blocks navigation menu” implemented in #10545 is a nice-to-have and may help in some workflows but doesn’t fully address the issue. The fundamental problem still stands, not going to reopen this issue because there's now #11581 with a detailed proposal to improve keyboard accessibility.