Improve keyboard shortcuts & a11y. (Still a bunch remaining)

[matrixbot]

Created by @ matthew:matrix.org.

See scrollback in #vector with Peter Vagner from Nov 1

[ara4n]

This is needed for accessibility to work properly

[pvagner]

I've forked matrix-react-sdk and vector-web in order to test this approach adding a few changes like this. It mostly works although after the latest changes introduced in Riot v 0.9 I am starting to believe we do need more help with accessibility as room sublists are very difficult to navigate and right pannel holding tab pannels such as member lists, files and notifications are impossible to reach with further tweaks.

[ndarilek]

Would like to bump this. I was sad to see this regressed after returning to Matrix and attempting to use the non-vanilla React client. First I had to reject an invite, which involved routing my screen reader to the correct div and simulating a click. I then had to do the same to enter a room. Once inside rooms, the interface is painful to navigate since it no longer seems to include landmarks. I think I submitted some PRs to the vanilla React client to use more semantic HTML tags (I.e. section/nav/aside, and a <ul/ for the chat.) These appear to have been removed, and the result is that I can't really use the web interface anymore.

Additionally, I suspect but cannot confirm that some controls are image-only, and are missing textual representations at all. If these controls aren't links/buttons then they can't be interacted with via the keyboard, but if they don't include an alt attribute, implicit text, or an aria-label then they almost certainly won't appear to the screen reader at all.

I have lots of respect for what you're doing, and as much as I'd like to roll up my sleeves and fix this, I feel like I already addressed some of these and my changes were rolled back. I understand it was accidental, but these tweaks really do make the difference between an easy-to-use client and one that I avoid. If we can restore the missing functionality and commit to keeping it, I'd be happy to dig in and submit additional accessibility-related PRs to make Riot the most accessible chat client out there. Slack will likely never fix their accessibility issues, so I actively recommend Matrix/Rocket.chat wherever I can precisely because you all try to create a more accessible experience. :)

[ara4n]

bumped priority. agreed that the current situation sucks :(

[pvagner]

@ndarilek I started to be afraid that you have moved to rocketchat exclusivelly. I haven't taken care of missing aria landmarks and proper list items in the chat output area however I am trying to keep interactive controls focusable and labelled in my forks.
I have it on a web server at https://matrix.pvagner.tk if you would like to see somewhat improved version in terms of accessibility.
What I am afraid we need help is popup menus. We need to implement javascript focus handling and aria-active-descendants I assume. Similarly for the room sublists.

[ndarilek]

Hmm, any chance we can get your forks upstreamed as PRs?

[pvagner]

@ndarilek I think it needs more work. I would preferably like to have someone sighted to at least look at it whether changing divs to buttons does not ruin the overal design before subbmitting it as a PR.
Additionally I've briefly chatted about my wish to make Riot more accessible with @ara4n so I assume people at the core team might be aware what we are trying to do here. If we have no one else to help with coding, then I wish more disabled people would join us by at least acknowledging we would be happy to have fully open decentralized and accessible messenger. Note I am also doing similar things with Riot Android but I am also not quite done with that either.

[ndarilek]

I understand that, but if you submit the PR then someone sighted can look over the changes as part of the review process, right? Also, if the element-based controls look different than their div counterparts, that's just a CSS tweak which designers could make on the Riot end. Basically, I was about to give up and change all the divs, but I don't have a sighted person to look that over so I'd be in the same boat. I'd think that getting part of the job done would be appreciated, and might inspire someone else to finish it.

[pvagner]

Okay, I'll wait a bit to find out whether we have made enough noyse these days if someone will try to join us. If not I will update to the head revisions during one of the upcoming weekends and send it as a pull request.

[ndarilek]

Thanks! Do you also address the noisy read receipt icons? If not, I'll take a crack at those once your PR lands. Thanks for working on this.

[pvagner]

I haven't yet found how to make them less anoying but have access to that information. I don't want aria-labels on the messages as some of the screen readers are not rendering aria-labels as a content e.g. orca with Firefox on linux.
Additionally there are more graphical only interactive controls associated with each message for example popup menu activation and encryption status which we do also need to be able to get properly accessible. Ideally I would like to avoid losing features as a trade of for better accessibility. I see accessible popup menus as a higher priority since this might become a building block for this in the future.

[ara4n]

@pvagner, @ndarilek: @JaniM did some work on this a few days ago, trying to replace all the divs with buttons. However, fighting against the default CSS for buttons seems to have been a nightmare.

Can anyone confirm whether <a role="button" onClick={...}/> is a good enough solution?

[ndarilek]

Two issues: 1. Please ensure that the button has `tabindex="0"` so it is keyboard-focusable. 2. Please ensure that the button responds to the Space and Enter keys for simulating clicks. I think that, if you do those two things, it should work. Maybe package it up in a React `<Button/>` component using a `<div/>` underneath, but with the tabindex/keyboard behavior included so it doesn't get missed for future buttons. This would also let us make any additional modifications in one place and have them reflected across the codebase. Thanks!

[pvagner]

Hello, By saying role"button" you are saying this control behaves similar to button, let's present it accordingly. So if you cant change <div>this is a button</div> into <button>This is a button</button> you will have to also implement onKeyDown and in some cases some other things in order to implement proper keyboard controls.

[ara4n]

so @JaniM fixed this up. @pvagner, @ndarilek - thoughts on whether riot.im/develop are an improvement would be most welcome. Going to close for now and reopen if needed after comments.

[ara4n]

(thanks @JaniM!!)

[pvagner]

Indeed thanks @JaniM this is very functional. It's a pity I haven't discovered this earlier. It's almost a month since you have pushed this.

Unfortunately there are some edge case and follow issues to be addressed:
I can point out some of them right now, however perhaps I have to fill them individually.

• When the left pane room lists are collapsed in the list of rooms perhaps individual items representing rooms need an aria-label because now only word DM is exposed to ATs.
• Buttons in the bottom left menu such as settings, room directory and all the others need aria-label or other technique to make them better accessible. Currently these can't be activated by using the keyboard alone. These buttons do have a label however the label is only shown on mouse over thus not shown at all for keyboard only users.
• There might be some left overs that are still divs. So far I was only able to notice Leave room button shown when clicking Settings button in the room header.
• When the member list is completelly collapsed room header has an image only button to show it. This perhaps needs an alt description or aria-label so people will understand what it does.
• Buttons next to the input field which are used for initiating voice / video call or file upload don't appear to be reachable via keyboard. I am not sure what they are missing my guess that they need either tabStop=0 or an alt text.
• We need to find out how to activate popup menu on an incoming message.
• We need to be able to switch side pannel from displaying member list to files and recents and / or the other way round. This is not accessible via keyboard.
• When the room search is showing we need to turn This room/All room toggles into radio buttons. I think similar technique can be used, however the role should be different.
• There is an image only button saying Search when the search is showing. This needs proper role as well.
• In the room statusbar Scroll to the bottom button appears when there are new messages in a room. This is a clickable image. It should also be turned into accessible button the same way most of the buttons have been tweaked recently.
• Text formatting toolbar is missing all the accessibility related awesomeness @JaniM has prepared as a general solution for accessible buttons.
• Both jump to last unread message bar and allow widget access dialog are confusing. I think there is a clickable button at the top, then there is an alert text and there is one or more buttons. Are those clickables at the top also actionable?
• Tabbing to a hidden link for each event in the messages timeline is confusing. Should that link be there at all and what's its purpose? For copying there is permalink in the individual event's menu. If the link should be there perhaps we may consider setting its text to a hard space or some other invisible character and set its title attribute to a string that would hint screen reader users to its functionality there.
• There is an unlabelled button on the left pane next to the filter rooms entry. Apparently it's used for collapsing / expanding that pannel.
• There are still some texts that can't be translated. E.G. image alt texts on the homepage and warning icon on a You are not receiving desktop notifications. There might be more however I haven't recognized them so far.
• Expand button for multiple events such as X has joined and 2 others left is not accessible and its text is not translatable.
• Rewoke widget access dialog has its text exposed as alert. All the other Riot dialogs I have seen are missing that.
• We need to reconsider if setting alt text of all avatar images is helping accessibility. There are some places e.g. right pannel member list and messages timeline where this adds unecessary extra verbosity as the surrounding text holds the same information.
• When changing Riot language via Settings UI list of languages is not keyboard accessible.
• It would be nice to consider marking typing indicator as an aria-alert so we can notice when someone is typing.
• It's not a very good practice to disable, remove or hide focused DOM elements e.g. after clicking. I have just noticed that for example when clicking the Settings button on the room header, the button goes away and focus stays nowhere. It would be nice if the new dialogs / windows / sub trees set their initial focus upon its creation.
• Event tiles in a room timeline are missing accessibility related markup and functionality on the user avatar if there is no image and a default avatar is shown instead. The default avatar is still clickable so I'd say this needs adjusting.
• Buttons toggling right pannel content (Members, Files, Notifications) are missing alt text or aria-label as they appear unlabelled.
• When there is no image room avatars in the left pannel room list is missing aria-hidden as previously the autogenerated avatar containing first name is exposed.
• When conference call is in progress it should be more noticeable for people using assistive tools. I think modifying window title would be more noticeable or modifying input text place holder to say something like send message (unencrypted, active conference call).
• When adding a phone number (including the registration form) the list of countries with their country codes (e.g. +49 for germany, +44 for UK) is not accessible. It is not working like a list it cant be controlled by using the keyboard and it is missing a markup so screen reader users can use and recognize it as interactive list.
• Issue sending out read receipts (It's very unnatural to send read receipts for keyboard only screen reader users #5679)
• It is not possible to scrol right pannel content (e.g. notifications) using the keyboard - thanks @grahamperrin Improve keyboard shortcuts & a11y. (Still a bunch remaining) #302 (comment)
• Room sublist items and message tiles in the main content timeline have some kind of dropdown arrow that displays a popup menu when hovering with the mouse. This is not useable with the keyboard nor it is accessible to screen reader users - thanks @pafcu Improve keyboard shortcuts & a11y. (Still a bunch remaining) #302 (comment)

Again some highlights in no particular order:
Incoming messages are turned into list items as @ndarilek proposed a while ago.
Users in the member list are focusable and activatable using the keyboard, this was not possible earlier.
User menu is also fully accessible it is possible to start new chat, change power levels and other moderation functions.
Room header buttons are accessible, this is also new.
Dialog buttons, buttons found in the settings and all the context buttons I have noticed so far are accessible.
Notifications such as the desktop notifications toggle, Riot update are also accessible.

[ara4n]

@pvagner thanks for all the feedback - and sorry that this has been stuck out on /develop. we're doing a release today however. reopening to cover all the accessibility bulletpoints enumerated above.

[pvagner]

I have found another nifty example in the riot code to explain my thinking.
In the TopLeftMenuButton component we render an AccessibleButton. That button has aria-label set to text "Your profile". However the button has a child element inside which includes user's display name.
The fact you have added aria-label saying "Your profile" is not an accessibility issue but an UX issue in general. Now screen readers present this as "Your profile button" if the aria-label would get removed it would be presented like "%{displayname}s button" which is still accessible.

Related to the TopLeftMenu it self. It's a HTML list with individual list items set as focusable by adding tabIndex=0. Other than that it has no other accessibility features. So items can be focused by navigating using tab and shift+tab keys however it would be more natural if using up and down arrow keys was implemented to move from item to item if it should resemble a desktop style popup menu.
For screen reader users there is no way to activate individual items in this menu other than emulating mouse clicks.
To improve that we'd need similar component to AccessibleButton. The difference to AccessibleButton would be that the role is "menuitem" for the items and role="popup" for the menu container. TabIndex should be -1 and focus management has to be implemented if it has to behave like a desktop style menu. See this for more discussion on a popup menu. Also it should be dismissable by pressing the ESC key. https://stackoverflow.com/questions/41141247/aria-role-menuitem-for-a-or-li

[turt2live]

Thanks for the detailed information - I'm also hiding in #a11y if it's ever easier to send a message there. I'm personally on a different project for the next couple weeks, but will keep the keyboard accessibility in mind and try to come up with better solutions to our accessibility problem. Once I'm back in riot-web land, I'll get them on /develop and visit #a11y for review.

[ara4n]

so @t3chguy and @turt2live have just landed another wave of screenreader work following feedback by @jcsteh at mozilla. feedback would be very welcome as to whether the situation has improved by testing riot.im/develop

[ndarilek]

Nice! I assume you know about the various buttons that still remain unlabeled, so I won't comment on those. One minor nit: when announcing rooms, you don't necessarily need to say "It has 5 unread messages." Specifically, the "it has" can probably go, as a) it is fairly intuitive and b) it slows down interaction a bit. Think of it like presenting any other button or visual control. You don't necessarily have to explicitly spell everything out, as users generally quickly figure out intent after a few interactions. Put differently, "5 unread messages", "5 unreads" or even simply "5" is pretty easy to figure out after an interaction or two, and the "it has" just takes up more audio space. Thanks again, this is looking good.

[t3chguy]

@ndarilek that is super valuable feedback, this is my first time doing any form of screen reader accessibility work and will take it all on board

[ptman]

Shortcut discoverability is low. Please document https://github.com/vector-im/riot-web/blob/develop/docs/shortcuts.md . Or maybe we need a popup (from ^? or ^h) to show them, like gmail, grafana, ...

[pvagner]

Hello,

I have checkmarked items in my old recap comment I think are now solved.

I have noticed these improvements:

• Settings is now much more accessible. Tabs are exposed as buttons, toggles are exposed as checkboxes. What needs more attention in the settings are lists of countries or list of languages. When choosing country prefix while adding phone number, list of countries is not keyboard navigable. The same is true for the list of supported Riot languages on the general tab. These are very minor points I'd say.
• Left pannel is fixed so rooms expose their names and unread count and it is possible to inwoke the popup menu. What is not yet possible by using the keyboard only is expanding or collapsing categories. For example if I collapse rooms by clicking with the mouse there is no way to bring that back by using the keyboard only.
• Suggested rooms at the top don't have names and are not keyboard accessible along with the Explore button.
• Popup menus are now accessible as a series of buttons. It is accessible from the keyboard and can be used with screen reader specific features. Focus management feels a bit unpredictable.
• It's nice to know auto complete improvements are working with new message editor that is not enabled by default. We can enable it in Settings Under Labs.

If there are some spare resources you can dedicate to accessibility related work, I'd appreciate the following:

• Make sure navigating to all the chats and rooms is possible via keyboard on the left pannel.
• Try to give us screen reader users access to Riot exclusive features like read receipts, reactions, replies and message specific popup menu.
• Rethink focus handling in the timeline and rich composer. It is only possible to navigate out of the message editor by using screen reader specific features. Tab key is trapped for auto completion I guess.

This is really getting better and better all the time.

[ptman]

I'm unable to trigger the new keyboard shortcut dialog on a Finnish layout. Maybe I'm doing something wrong.

[t3chguy]

Its Ctrl + event.key=/ https://keycode.info/

[ptman]

@t3chguy yes, so that's Ctrl+Shift+7 on my layout and it doesn't trigger.

[t3chguy]

So keycode.info on shift+7 gives you a forward slash?

[ptman]

Correct. setxkbmap fi

[t3chguy]

Okay so this looks like it'll an issue elsewhere too where we check for "CmdOrCtrl" only so the "SHIFT" makes it fail. Will loosen it for this exact one.

[kittykat]

@pvagner would you be able to add an update on what outstanding items need to be resolved before this issue could be closed?

[pvagner]

@kittykat I have checkmarked things I think have been solved.
The whole app has changed since I have documented these so perhaps it would be acceptable to close this issue and move on fixing recently pointed accessibility issues or wishes.
For example if I can suggest some priorities, coming back to the issue with time line live regions where @Palid has left would be nice.
Also using same technique @t3chguy has used while making room list more accessible should get applied to the room member list.
Room directory is next on my radar as I think rooms can't be joined using keyboard alone.

[t3chguy]

@pvagner the member list is issue #11285

[Palid]

#9747 @kittykat that's the issue

[kittykat]

I'm going to close this issue now as #9747 and #11285 cover the two remaining points 👍