NVDA cannot navigate math content in Chromium

[cary-rowen]

This is most likely a Chromium regression, but it's unclear which version introduced it. I hope the community can provide more information. In the meantime, I will report this issue to the Chromium team.

Steps to reproduce:

1. Open the following Wikipedia page: https://en.wikipedia.org/wiki/Quadratic_equation
2. Use NVDA to locate the mathematical content on the page. You can use Single Letter Navigation commands.
3. Press the space key on the math content to try interacting with it.
4. Use the left/right arrow keys to navigate the math content.

Actual behavior:

• Step 3: Unable to enter interactive mode.
• Step 4: Unable to navigate or find the math object.

Note: Using object navigation (NVDA+numpad4/6), you can locate the mathematical content, but interaction is still not possible.

Expected behavior:

NVDA should be able to interact with math content in Chromium, just like it does in Firefox.

NVDA logs, crash dumps, and other attachments:

System configuration:

NVDA installed/portable/running from source:

Installed

NVDA version:

2024.4.1 (Alpha snapshot)

Windows version:

Windows 10 22H2 (AMD64) Build 19045.5131

Name and version of other software in use when reproducing the issue:

Chrome 131.0.6778.86

Other information about your system:

None

Other questions:

Does the issue still occur after restarting your computer?

Yes

Have you tried any other versions of NVDA? If so, please describe their behavior:

The issue can be reproduced in versions 2024.1, 2024.2, and 2024.3.

If NVDA add-ons are disabled, does the problem still occur?

Yes

Does the issue still occur after running the COM Registration Fixing Tool in NVDA's tools menu?

Yes

[cary-rowen]

Hi cc @NSoiffer
Can you take a look at this?

Many thanks

[NSoiffer]

It worked for me yesterday or the day before that on Chrome. I don't remember what the version was. Version 131.0.6778.71 and the just updated Version 131.0.6778.86 are not triggering the calls to MathCAT. Everything is working in Firefox. Edge Version 131.0.2903.51 has the same problem as Chrome (not surprisingly). The problem is also there in Chrome Canary Version 133.0.6849.0.

I suspect the stable build on Nov 12 (131.0.6778.70) was working. However, Google doesn't make it easy download old versions and I failed in my attempt to download that version.

Note 1: on my Windows 10 computer, there is no speech period, let alone the ability to navigate it.

Note 2: NVDA 2025.1 alpha builds fails for navigation of math with MathCAT due to a problem in NVDA's braille code. That's a separate issue that I hope to file a bug report on in the next few days. It is not a problem for 2024.4 (or 2024.4.1).

[Adriani90]

If you report this to Chromium as well, please post the link here so we can track it as well.

[SasnikOmega]

I have the same problem. Heres what I noticed.
Looking at detailed info about the current navigator object using NVDA f1, theres a property called "IAccessible2 attributes".
Usually its value looks like this:
IAccessible2 attributes: 'display:block;tag:p;text-align:left;'. Or, in case of math objects:
IAccessible2 attributes: 'display:block math;tag:math;text-align:left;'.
In last versions of Chromium-based browsers though, NVDA considers math content itself as the part of IAccessible2 attributes. It looks like this:
IAccessible2 attributes: 'display:block math;tag:math;text-align:left;math:\n<mrow>\n <mi>x</mi>\n <mo>\\=</mo>\n <mfrac>\n <mrow>\n <mrow>\n <mo>-</mo>\n <mi>b</mi>\n </mrow>\n <mo>±</mo>\n <msqrt>\n <mrow>\n <msup>\n <mi>b</mi>\n <mn>2</mn>\n </msup>\n <mo>-</mo>\n <mrow>\n <mn>4</mn>\n <mo>\u2062</mo>\n <mi>a</mi>\n <mo>\u2062</mo>\n <mi>c</mi>\n </mrow>\n </mrow>\n </msqrt>\n </mrow>\n <mrow>\n <mn>2</mn>\n <mo>\u2062</mo>\n <mi>a</mi>\n </mrow>\n </mfrac>\n</mrow>\n;'
(Source
Unfortunately, thats all I can say. Hope that helps though.

[NSoiffer]

FYI: here is the Chromium bug report: https://issues.chromium.org/issues/380161205.

[cary-rowen]

Thanks @NSoiffer

[cary-rowen]

Thanks @SasnikOmega

The information you provided may be useful and I will update the progress of the original bug here.

[aleventhal]

I'm guessing that Chrome needs to strip out the newlines so that it doesn't break parsing of the object attributes. It wouldn't have been an issue when NVDA retrieved the math through ISimpleDOMNode::get_innerHTML because it doesn't need to break that string up into multiple attributes.

[cary-rowen]

Yes, thanks to the Chromium engineers for working on this.

Let's keep an eye on the canary version.

[NSoiffer]

I've been in contact with the developers and got this response:

It seems that the issue is that we return E_NOTIMPL from ISimpleDOMNode::get_attributesForNames() now. If it's changed so that we return S_FALSE or S_OK, then the math is spoken as expected. We don't even have to return the attribute, but we can't return an error.

I checked NVDAObjects.IAccessible.ia2Web.Math._get_mathMl and indeed, the call
node.attributesForNames(1, attrNames, namespaceIds)
returns

DEBUGWARNING - NVDAObjects.IAccessible.ia2Web.Math._get_mathMl (16:05:01.897) - MainThread (5344):
Error retrieving math. Not supported in this browser or ISimpleDOM COM proxy not registered.
Traceback (most recent call last):
  File "NVDAObjects\IAccessible\ia2Web.py", line 341, in _get_mathMl
    attr = node.attributesForNames(1, attrNames, namespaceIds)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "C:\Dev\nvda\.venv\Lib\site-packages\comtypes\_memberspec.py", line 523, in __call__
    return self.fget(self.instance, *args)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
_ctypes.COMError: (-2147467263, 'Not implemented', (None, None, None, 0, None))

There is some trickery in generating the COM call, so I don't know if this is fixable via an NVDA change or not. Does anyone more knowledgeable know?

FYI: I checked JAWS and JAWS doesn't have a problem with Chrome.

[cary-rowen]

Chromium may roll back some changes at a later date to restore the math reading experience for NVDA users.
But @NSoiffer's comment is still worth noting.

cc @jcsteh
cc @michaelDCurran

Can you take a look at @NSoiffer's comment and make some comments?
I can talk to the Chromium engineers directly, but I don't quite understand what NVDA is supposed to do.
JAWS in particular is not affected by this change.

[NSoiffer]

I wrote my earlier message a little hastily as I needed to make dinner. The obvious thing to do is to catch the error. I added try/except:

			try:
				attr = node.attributesForNames(1, attrNames, namespaceIds)
			except COMError:
				attr = None

and the math was back to working. Then I tried to be more specific and catch just E_NOTIMPL, but messed something up and now .\runnvda.bat is stuck running something other than the current code (some version with different debug messages). I ran into this before, but can't remember the easy solution and I need to call it quits for today. Maybe someone can take the ball from here.

[aleventhal]

Fixes should be on the consumer side imo because E_NOTIMPL is the correct thing for Chromium to return for this API method we no longer implement. That said, I recognize it's a change of behavior in Chromium, so if the NVDA devs believe we should return S_OK or S_FALSE instead for now as a practical matter, lmk and I will do that. I'm extremely sorry about the regression for users and also want to get it fixed with urgency.

…

On Tue, Dec 3, 2024, 1:16 AM NSoiffer ***@***.***> wrote: I wrote my earlier message a little hastily as I needed to make dinner. The obvious thing to do is to catch the error. I added try/except: try: attr = node.attributesForNames(1, attrNames, namespaceIds) except COMError: attr = None and the math was back to working. Then I tried to be more specific and catch just E_NOTIMPL, but messed something up and now .\runnvda.bat is stuck running something other than the current code (some version with different debug messages). I ran into this before, but can't remember the easy solution and I need to call it quits for today. Maybe someone can take the ball from here. — Reply to this email directly, view it on GitHub <#17421 (comment)>, or unsubscribe <https://github.com/notifications/unsubscribe-auth/AAKQAZUWWVE3RBXDIYBGUOL2DVEFDAVCNFSM6AAAAABSFYTM4CVHI2DSMVQWIX3LMV43OSLTON2WKQ3PNVWWK3TUHMZDKMJTGY2TIMBZHA> . You are receiving this because you commented.Message ID: ***@***.***>

[CyrilleB79]

@seanbudd, @gerald-hartig given the last comments, could NV Access quickly indicate if they plan to provide a quick fix themselves or if they need Chromium (@aleventhal) to temporarily change again the behavior of the API.

This is an important blocking issue for whoever is working with math content and has no choice of the used browser.

Also, if there is a workaround or an add-on/script to have this fixed temporarily, please let it know. Thanks.

[cary-rowen]

cc @seanbudd
cc @gerald-hartig
Can you please comment here?

[michaelDCurran]

NV Access and Google are collaborating on this and have been speaking offline. We'll share updates when they are available.

[cary-rowen]

Fixed in 133.0.6878.0 canary.
Thanks @aleventhal

[Adriani90]

As far as I understand the fix in Canary is only temporarily, so on a longer term a fix on NVDA‘s side is needed, is this correct?

[NSoiffer]

I've committed the fix/PR.

Apologies for the delay: my desktop has COM issues (I think too much mucking about with COM stuff) and stopped handling ISimpleDOM. I have a new laptop and after a painful process of getting it all setup, I was able to essentially add the code I mentioned above.

With the fix, the math goes from broken to working in Chrome.

[aleventhal]

Thanks @NSoiffer . We also merged the fix into Chrome 132.0.6834.46, the current beta.

Chrome 132 will be released to stable in mid-January.
Early Stable Release Jan 8
Stable Release Jan 14

Users can force the update starting January 8 by visiting Help -> About.