-
Notifications
You must be signed in to change notification settings - Fork 11
cuaaddoninternals
Author: Joseph Lee
Computer screens, or for that matter, apps such as web browsers and web pages, are filled with graphical controls. The intention is to let users quickly glance the screen and use devices such as keyboards and mice to interact with controls on screen. But what about screen reader users, or for that matter, how can new screen readers interact with controls on screen?
One approach to teaching new users about control interaction strategies is context-sensitive help. Screen readers such as JAWS provide a help command that offer users information on where they are and helpful commands to interact with the focused control. For example, when focused on a link, context-sensitive help in JAWS (JAWS Key+F1) identifies the control and offers tips to interact with a link such as pressing Enter to activate it.
Despite being a popular screen reader for Windows users, NVDA does not offer a native context-sensitive help feature for controls outside of NVDA (NVDA does open the user guide to relevant sections when pressing F1 from its own controls such as settings). Enter Control Usage Assistant, an add-on that adds a command to open context-sensitive help message for the focused control. In NVDA Add-on Internals: Control Usage Assistant, we will learn about how the add-on came to be and its mechanics.
Control Usage Assistant, as the name suggests, assists users with interacting with the focused control through context-sensitive help messages. This is done by pressing NVDA+H (help) from the focused control which opens a browse mode document where NVDA provides tips on interacting with the control in question.
In October 2012, an issue was submitted that requested context-sensitive help functionality. Back then I (Joseph Lee) was a newbie to NVDA project, mostly concerned with translations. I heard about the context-sensitive help feature suggestion but didn't come back to it for several months.
In early 2013, I decided to venture into add-ons, and the first add-on I worked on was Control Usage Assistant after hearing rumors about a feature suggestion made a few months earlier. The first add-on release defined a control roles dictionary that mapped role identifiers to help messages (back then, roles were internally stored as integers). From the first release, NVDA+H command was defined to present help messages, although it only presented help text in speech and braille. Since roles were integers, several integer ranes were defined to presnet variations of existing help messages in contexts such as browse/focus modes in web browsers and in different programs such as Microsoft PowerPoint.
Control Usage Assistant went through a dramatic change in 2019. Not only roles to help messages map was completely rewritten (uses class names instead of role integers), a browse mode document was employed to present help messages. Another change took place in 2021 due to change of control roles from integers to enumerations (enum) in NVDA 2021.2. In 2022, a new maintainer now maintains this add-on.
Control Usage Assistant is a global plugin consisting of three components:
- Help message presentation (init.py), responsible for displaying help messages upon request.
- Role-based help messages (controltypeshelp.py), records help messages for controls based on their roles.
- Messages from specific NVDA objects and apps (nvdaobjectshelp.py), presents help messages for specific NVDA objects such as table cells and in apps such as PowerPoint.
As described below, it is not merely enough to define help messages for different roles and objects - sometimes NVDA checks additional things such as whether browse mode is on or off before presenting the most appropriate message for the given situation.
From its beginning, Control Usage Assistant relies on control roles to present help messages. For example, help message for a button would be shown as, "press Space to activate this button." These messages come from a map defined in control roles help module, with each role mapped to a help message.
Sometimes it is useful to provide more context when working with specific controls found in various apps. For example, on Windows 10 search field, it isn't enough to tell the user to type text, so a message about the purpose of the search field is announced, namely asking users to enter search terms.
Two maps are used to house help messages for controls and apps:
- Control help messages: a map of NVDA objects to contextual messages is provided to handle cases such as search fields and table cells.
- App-specific controls: a map of controls found in apps is provided to provide messages in cases such as reviewing slides in PowerPoint.
To make it easy to extend the add-on, help message presentation is divided into three parts:
- Actual help message script.
- Help message retriever.
- Special situations.
When NVDA+H is pressed, a list of help messages is gathered in the following order:
- If the focused control provides a special method that returns a help message for itself, whatever the object says will be used.
- Help messages for objects is retrieved through method resolution order (MRO). For example, if focused on a PowerPoint slide, NVDA will check if the object class is recorded in objects help messages map, and since it is present, the recorded help message will be retrieved.
- If MRO method fails, control role is consulted. This is the point where NVDA may check for special situations (see below).
- If no help message is present, NVDA will add an error message.
- NVDA will add a reminder message on closing the help screen.
The contents of help messages list (if any) is then concatenated and then presented as a browse mode document.
One special situation NVDA checks is browse mode. If focused on a form field such as a radio button while browse mode is active, NVDA will add a variant of control role messages for the focused control. If focus mode is active and if a document is focused, NVDA will add a reminder about switching to browse mode.