Support message text-to-speech through browser.SpeechSynthesis API

[sosnik]

Your use case

What would you like to do?

I would like native support for text-to-speech playback of incoming text messages in the Element client.

Why would you like to do it?

I am legally blind. Text-to-speech helps me (and other people with print disabilities) keep up with chat.

How would you like to achieve it?

Element is a web app, which means it has access to the [browser.SpeechSynthesis API](https://developer.mozilla.org/en-US/docs/Web/API/SpeechSynthesis). TL;DR the API allows web pages to pass text along to whatever TTS engine is installed on the user's machine, and the OS does the rest.

You can see a working example on Discord. To enable TTS on Discord for all incoming messages, go to the User Menu -> Accessibility.

Have you considered any alternatives?

Screen readers can theoretically be used to read incoming chat, but practically this is almost impossible. A solution native to the client is best because:

• You don't need to have the chat app open in the foreground and focused
• The chat app can target which content needs to be read - a screenreader is liable to read everything on screen, including mouseover actions and the title bar of the application. Several times.
• TTS integration will, naturally, target incoming messages. A screenreader is likely to read from the top of the screen, or from begining of chat history.

Additional context

I note that element-hq/element-meta#1626 suggested a specific /tts command for TTS playback of certain messages. While this has its uses (and nuisances), my suggestion is for client-side TTS playback rather than a /tts command.

[germain-gg]

Thank you for sharing your use case here!
There's been a fair amount of work in the past to make Element usable with a screen reader. There is also an interesting room with a few folks that are involved in a11y in Element, https://matrix.to/#/#element-accessibility:matrix.org

[MadLittleMods]

I am familiar with using ARIA live regions for this kind of thing. You can define how you want the screenreader to announce new or changed elements on a page. role="log" aria-live="polite" aria-relevant="additions text" will announce new messages and edits to messages for example.

Update: It looks like we already have an ARIA live region in place but it could use some work: #10952

[sosnik]

I am familiar with using ARIA live regions for this kind of thing. You can define how you want the screenreader to announce new or changed elements on a page. role="log" aria-live="polite" aria-relevant="additions text" will announce new messages and edits to messages for example.

Update: It looks like we already have an ARIA live region in place but it could use some work: #10952

I'm glad we have this, but it doesn't address "playback in the background" that is possible by using browser.speechSynthesis. Here are some concrete examples of where background TTS may be useful:

• Idling in a support room
• Small group discussion room with irregular participants
• Moderation