Consult: space/enter keyEvent is swallowed by NVDA

[mo13yu]

I'm working on a project that within a QT application, there is QWidget loads an html. And I need when using NVDA tab navigate to a button inside of the html, if user presses space/enter, not only the button's onClick handler should be triggered, but also the parent widget should handle the keyPressEvent. However I observed the parent widget doesn't get the keyPressEvent.

I've already find out that is because browse mode vs. focus mode are automatically toggled when on web page. Here(#4903) says: "when in browse mode, the enter key is overridden to activate the control using mouse clicks." Could someone show me which part of code is? (seems my parent widget doesn't get mouseEvent either)

I did try manually toggle to focus mode everything works fine. And I also read this from user guide: "NVDA will automatically switch to focus mode if you tab to or click on a particular control that requires it."

My questions are:

1. Is there any programmatic way that I can specify this html button requires focus mode?
2. Is there any programmatic way to toggle focus mode/browse mode in code?
3. What are the html controls that requires focus mode?
4. Is there any way to check what mode is NVDA currently on? (As they are auto-toggled, so I need a way to specify)

Thanks!

[LeonarddeR]

This is a very interesting issue. First of all, it could be good to know how QT renders this HTML (i.e. does it use MSHTML, GECKO or Chromium?). Could you provide developer info (insert+f1) after focusing the particular button?

I wonder, couldn't you make the parent widget react on an event that is triggered by the onclick?

Is there any way to check what mode is NVDA currently on?

You can distinguish the mode by the sound played when it activates. In NVDA's browse mode settings, you can disable Audio indication of focus and browse modes, in which case a switch will be spoken by speech.

[jcsteh]

@mo13yu commented on Jan 6, 2018, 12:43 PM GMT+10:

seems my parent widget doesn't get mouseEvent either

I assume you mean parent Qt widget, not HTML widget? That's tricky. In many cases, NVDA will dispatch a click event using the accessibility API, which will be handled internally by the browser engine. So, it won't be intercepted by Windows (and thus probably not passed to Qt). I guess you'd need to overide the click event in HTML to somehow notify the Qt widget yourself. Or is there some way to monitor web DOM events from the Qt widget?

1. Is there any programmatic way that I can specify this html button requires focus mode?

No. Buttons should never require focus mode from a user perspective. Only controls that need keyboard interaction beyond clicking (e.g. text boxes, sliders, list boxes) cause a switch to focus mode.

2. Is there any programmatic way to toggle focus mode/browse mode in code?

No. You could use role="application" to forceably disable use of browse mode, but that would mean the user wouldn't be able to use browse mode.

3. What are the html controls that requires focus mode?

See above.

4. Is there any way to check what mode is NVDA currently on? (As they are auto-toggled, so I need a way to specify)

No. We can't expose any API to web pages.

[vadim-on-github]

it's necessary for the web developer to know whether an element was activated using a mouse click or a keyboard's Enter/Space key, even in Browse Mode

[danzen]

We need Browser window key events to fire when NVDA is in browse mode as well. Our canvas framework at https://zimjs.com needs to deal with its own enter events. Currently, it works well in Form mode or if a shift or ctrl ENTER is used. But that is not ideal. If you could indicate the likelihood of this being adjusted that would be great. We have an upcoming government test... Thanks.

[Adriani90]

cc: @feerrenrut

[feerrenrut]

it's necessary for the web developer to know whether an element was activated using a mouse click or a keyboard's Enter/Space key, even in Browse Mode

@vadimhp Why is it necessary? What is the semantic difference for a user activating an element via mouse click or via keyboard? How would you like to expose that semantic difference to an end user?

If you could indicate the likelihood of this being adjusted that would be great.

@danzen Very unlikely at the moment. Please provide more information about what you are trying to achieve.

[danzen]

Thanks for the response, @feerrenrut. We are a Canvas framework and so click events are put on our canvas objects to handle click interactivity. HTML tags are then placed behind the canvas in the same place as the canvas objects. When a user hits enter we capture the enter key being pressed and call the same function that the click events call. If we can't capture a key event in our framework, we can't do this. Any advice would be helpful.

[feerrenrut]

@danzen perhaps you can provide a link to an example, I had a look at the examples on the website linked earlier but could not focus any elements in the canvas with NVDA. If the goal is to mimic a mouse click, then the standard clickable activation should work, but I might be misunderstanding the situation.

[danzen]

Hi @feerrenrut - here is what we did a while ago. We have made a few changes since but mostly to work with Adobe Animate. https://zimjs.com/accessibility/ - sorry, should have posted a more specific link earlier.

[thomasrye]

I wanted to chime in here. #8317 was much more of a direct example of this in my opinion, but it has been merged with this one.

While we wait for some kind of resolution, does anyone know what experience NVDA users do when they come across a button that does not seem to respond to Enter or Space? Are they just used to immediately toggling to Forms mode or trying Ctrl+Enter to make it work and they move on with their task or does it create a lot of confusion for the user?

Just for further input, our particular case involves a <button> that gets it's click event from angular/typescript. The browser (Firefox) recognizes that there is an event on the button, but apparently it is not enough for NVDA to recognize and allow Enter/space alone to active it.

[XLTechie]

Thomas Rye wrote:

While we wait for some kind of resolution, does anyone know what experience NVDA users do when they come across a button that does not seem to respond to Enter or Space? Are they just used to immediately toggling to Forms mode or trying Ctrl+Enter to make it work and they move on with their task or does it create a lot of confusion for the user?

It can introduce quite a level of confusion, simply because the user doesn't really know that nothing happened. Maybe some sort of HTML menu opened or expanded, that NVDA doesn't speak natively. Maybe something happened elsewhere on the page, and NVDA doesn't know it. Take many of the controls on https://getresponse.com, for example. It's been a while since I tried to use that system, but after a list is created, many of the functions are operated by controls that open ajax dialogs or overlays that NVDA knows nothing about. So the user never really knows that nothing happened. In other cases, yes: other things can be tried to actuate the control.

[feerrenrut]

@danzen I tried https://zimjs.com/accessibility/radiobuttons.html

So far I'm not seeing a problem with NVDA, though here is some advice for zimjs, or at least the examples I looked at:

• Putting this section in a role=application section should make focus mode enabled by default, keys will be passed through directly, however you will have to do a lot more work to provide a user friendly UX. An alternative would be to actually handle the click, if you add onclick="alert();" to the div with id="myCanvasTab1" and press enter while it has focus (with NVDA running in browse mode) then the alert is triggered.
• The elements are all exposed as role="button" and the "name" is overloaded to include semantic information via aria-label. It would be much more friendly to provide the appropriate semantic information. For instance with the div that displays as radio button A id="myCanvasTab1" (I think the label should be A not eh by the way). This should be role="radio", checked state should be exposed with aria-checked="true", then you don't need to programmatically update the aria label with the semantic information.

See https://www.w3.org/TR/wai-aria/#widget_roles

I wanted to chime in here. #8317 was much more of a direct example of this in my opinion, but it has been merged with this one.

@thomasrye thanks for pointing that out. My gut reaction to that example is that you have a web application that has its own keyboard handling code. In this case role=application MAY be more appropriate, though you are forcing the user into a particular kind of experience (hopefully a good one), there is certainly a lot more to consider if going this route.

The browser (Firefox) recognizes that there is an event on the button, but apparently it is not enough for NVDA to recognize and allow Enter/space alone to active it.

This is interesting, and I haven't investigated. Is there a reason that the button doesn't have an onclick? Adding onclick="alert();" shows that pressing enter on the button creates an alert. That is, the button can be activated with the enter key while NVDA is in browse mode.

It seems I don't understand what it is you are trying to achieve here.

• You want to be able to differentiate between keyboard activations and mouse activations?
• You are limited by some framework constraint?
• Something else?

[thomasrye]

@feerrenrut Thanks for the response. (I'm late getting back to you.)

I think it is possible that there is some framework constraint. We are using angular and some data table component that makes the click handling less direct. I will try to look into that more

[mrMagic-web]

Thanks @feerrenrut. Your suggestion helped in my case. I have rapped entire block in role application and gave clickable elements correct button roles.

[Adriani90]

Is there any question still open to this issue? Or can this be closed?

[danzen]

Hi @Adriani90 - you can close it - and thanks all for the information. I think we fixed things some other way but if it comes up again we can re-open. Cheers.

[feerrenrut]

Thanks @danzen

[ColmShannon]

@feerrenrut Hi,

I've read the comments here and on #8317 after encountering a similar issue

We have an Angular dropdown component that uses HostListener to listen for keyboard clicks (arrow keys, Enter, Esc etc) to allow for keyboard accessibility.

We've just found that this doesn't work correctly with NVDA, and it seems related to this issue. Basically NVDA is blocking the keyboard events from reaching the component.

I've tried wrapping the component in role=application but this seems to do more harm than good.

After reading the above, I have a few questions.

1. Why does NVDA do this in Browse mode?
2. The JSFIddle shared in Keydown Events for Enter Key Not Firing #8317 https://jsfiddle.net/ubxvtgq4/8/ seems a valid example of an issue with NVDA. That bug was closed as a duplicate of this, with the point made that it works in Focus mode, but it doesn't seem to me that it was resolved. The suggestion to use Focus mode seems redundant, when if I understand the modes correctly, it's reasonable to expect it to work in Browse mode.
3. Given the age of this ticket, and assuming that this can't or won't be changed, is the current best recommendation from NVDA team still to use role=application?

Thanks in advance for reading and apologies for the comment on such an old ticket.

[feerrenrut]

1. Browse mode makes it easier for AT (Accessibility Technology) users to navigate a document. It provides easy, single button shortcuts for navigation. For example, next heading: 'h', next link: 'k' etc. Applications / Forms / Focus mode is prioritizing interaction.
2. The approach in the linked JS fiddle is bypassing standard abstractions to handle a button press. I'd recommend using the onclick handler (I have created an example on codepen - full view for testing). Of course, if it is important to be able to differentiate between a mouse click and keyboard input then there are many more considerations required. It may also interest you to know that the accessibility API (MSAA) exposed by the browser has a "default action" (accDoDefaultAction) which the browser maps to onClick. You can test this with my codepen example and inspect.exe (from the Windows SDK) via the 'action' menu.
3. This is not likely to change, it is fundamental to how our users expect to interact.

We have an Angular dropdown component that uses HostListener to listen for keyboard clicks (arrow keys, Enter, Esc etc) to allow for keyboard accessibility.

Here is an angular select box example to create a dropdown list which works as expected with NVDA.
To test this:

• Ensure you are in browse mode (NVDA+space) listen for "boop" or "browse mode" depending on your settings.
• press f until you hear "combo box collapsed"
• press alt+down arrow to expand the combo box NVDA will automatically swap into focus mode to interact with the combo box

[ColmShannon]

@feerrenrut we've mostly resolved our issue. Wanted to share what we found here incase it helps others.

In our case, we were missing the appropriate roles on our elements (e.g. role="menu"). Once we added these in the correct places, NVDA no longer seemed to swallow the keyboard event.

[warriotox]

Hello @feerrenrut I am sorry to re-open this, but I have just spent over 5 hours trying to figure it out and I can't seem to find a working solution for having NVDA in browser mode and being able to trigger separate JS logic for 'click' and 'keydown' events. I have seen your reply (quoted below) from when the issue was raised last time, but the CodePen you have shared is missing keypress logic completely (only when the last JS line is uncommented it starts triggering, but even then both events fire at the same time resulting in unwanted behavior).

2. The approach in the linked JS fiddle is bypassing standard abstractions to handle a button press. I'd recommend using the onclick handler (I have created an example on codepen). Of course, if it is important to be able to differentiate between a mouse click and keyboard input then there are many more considerations required....................

What I am trying to achieve is having a custom button (p tag with role=button and tabindex=0) act as a button with event listeners attached through javascript instead of through inline attributes. The logic I have works fine even when NVDA is enabled, but under the hood it's not working as expected. The key swallowing reported by others doesn't occur for me, but my experience was even stranger, because using a keyboard actually triggers a 'click' event when NVDA is active! When NVDA is not running it works as expected - triggering 'keydown' event.

Please see my JS fiddle here - https://jsfiddle.net/1m504ub6/1/
Steps to reproduce the incorrect event behavior:

1. With NVDA not running click the button with your mouse. You should see 'Countx: 0, event type: click'
2. Focus it with keyboard and press enter. you should see 'Countx: 1, event type: keydown'
3. Enable NVDA and click it with the mouse. you should see 'Countx: 2, event type: click'
4. Focus it with keyboard and press enter. you should see 'Countx: 3, event type: keydown', but in this case it actually displayes 'Countx: 3, event type: click'!

Could you please shed some light onto this strange behavior? Why and how could possibly NVDA change the event that is observed?

[feerrenrut]

but the CodePen you have shared is missing keypress logic completely

The point of that example was to show that "button click" can be triggered via the keyboard. Try pressing tab until the "press enter" button has focus, then press the enter key. The click handler will be called.

only when the last JS line is uncommented it starts triggering, but even then both events fire at the same time resulting in unwanted behavior).

I don't recall if that comment was left there for any intentional reason. You would need to remove the onclick from the HTML if uncommenting that. Bear in mind doing this will only demonstrate the issue.

The key swallowing reported by others doesn't occur for me, but my experience was even stranger, because using a keyboard actually triggers a 'click' event when NVDA is active! When NVDA is not running it works as expected - triggering 'keydown' event.

I touched on this in my last comment:

It may also interest you to know that the accessibility API (MSAA) exposed by the browser has a "default action" (accDoDefaultAction) which the browser maps to onClick. You can test this with my codepen example and inspect.exe (from the Windows SDK) via the 'action' menu.

Expanding on this, when NVDA tries to "activate" a button, it uses the accessibility API default action accDoDefaultAction. The browser maps this to "onClick".

There seems to be some confusion about what level of abstraction you are trying to work at, the web standards don't help to clarify. A button has a single semantically interaction, it can be pressed, whether that is done via a mouse click, a touch, a key press, or programmatically shouldn't matter to the underlying handler. If you are trying to intercept all of these different input types you are operating at a lower level of abstraction, there will be more to get right.

The short answer to the "strangeness" NVDA activates a button via the accessibility API, implemented in the browser, which programmatically calls 'onClick'.