Abbreviation extension should not inject markup in mathjax contexts

[mdbenito]

Assume we have defined an abbreviation

*[YAFR]: Yet Another Feature Request

Then the following markdown

Our function $f_\text{YAFR}$ does some things

will be expanded to (something like):

Our function $f_\text{<abbr title="Yet Another Feature Request">YAFR</abbr>}$ does some things

This will cause a maths renderer like mathjax to fail. It is actually rather common to add subindices to notation in maths that refer to abbreviations defined in the text, so I think it would be nice to have the extension not try to expand the text.

[mitya57]

Are you using any extension for rendering math, such as arithmatex or python-markdown-math?

These extensions should take care that math does not interfere with other markup. If you are not using them, then Markdown does not know that $ means start or end of math block, and does not apply any special treatment to it.

[facelessuser]

Yes, Markdown doesn't know anything about MathJax or LaTeX-style syntax, and consequently doesn't care about expectations related to them. As @mitya57 mentioned, you must identify and protect such syntax from the Markdown parser via an extension if you care to preserve them for some other parser.

[mdbenito]

Are you using any extension for rendering math, such as arithmatex or python-markdown-math?

Yes, we are using arithmatex, with the default config as recommended by mkdocs-material. I'm looking at diffs between our docs for v.0.9.2 without the error and those at develop with the error and I can't spot the problem at first sight.

In both cases, the math is surrounded by <div class="arithmatex"> but in one it's rendered and in the other not (and contains the expanded abbreviation).

The markdown_extensions config hasn't changed at all, neither has mathjax's. I'm afraid I'm in for a grueling git bisect session if I want to get to the root of this :/

However I have worked around the issue by changing the text, so I don't know when I'll look into it.

[facelessuser]

@mdbenito If you have an issue with Arithmatex, please create a Discussion over at: https://github.com/facelessuser/pymdown-extensions. Please provide specific, reproducible examples. I suspect this might be a user error, but without real examples, it is hard to say.

[mitya57]

It looks like I can reproduce this with both arithmatex and python-markdown-math:

>>> import markdown
>>> text = """
... Our function $f_\\text{YAFR}$ does some things
... 
... *[YAFR]: Yet Another Feature Request
... """
>>> markdown.markdown(text, extensions=["pymdownx.arithmatex", "abbr"])
'<p>Our function <span class="arithmatex"><span class="MathJax_Preview">f_\\text{<abbr title="Yet Another Feature Request">YAFR</abbr>}</span><script type="math/tex">f_\\text{<abbr title="Yet Another Feature Request">YAFR</abbr>}</script></span> does some things</p>'
>>> markdown.markdown(text, extensions=["mdx_math", "abbr"], extension_configs={"mdx_math": {"enable_dollar_delimiter": True}})
'<p>Our function <script type="math/tex">f_\\text{<abbr title="Yet Another Feature Request">YAFR</abbr>}</script> does some things</p>'

[facelessuser]

Oh, it looks like I didn't read too close as there is a reproducible example in the opening post 🤦🏻. And yep, it is reproducible. But this is not a Python Markdown issue.

It seems that maybe these extensions need to run earlier to avoid such issues.

[facelessuser]

I've created an issue for Arithmatex that I can look into when I have a chance.

[facelessuser]

Maybe there is a bug in Python Markdown. I'm getting the feeling that Abbr is not respecting Atomic strings.

Here is an example with block and inline math:

import markdown
text = """
Our function $f_\\text{YAFR}$ does some things

$$
f_\\text{YAFR}
$$

*[YAFR]: Yet Another Feature Request
"""
print(markdown.markdown(text, extensions=["pymdownx.arithmatex", "abbr"]))

We can see that Arithmatex intercepts them with the debug statements without abbreviations:

➜  pymdown-extensions git:(main) ✗ python3 example.py
MATH DEBUG:
f_\text{YAFR}

MATH DEBUG f_\text{YAFR}
<p>Our function <span class="arithmatex"><span class="MathJax_Preview">f_\text{<abbr title="Yet Another Feature Request">YAFR</abbr>}</span><script type="math/tex">f_\text{<abbr title="Yet Another Feature Request">YAFR</abbr>}</script></span> does some things</p>
<div class="arithmatex">
<div class="MathJax_Preview">
f_\text{<abbr title="Yet Another Feature Request">YAFR</abbr>}
</div>
<script type="math/tex; mode=display">
f_\text{<abbr title="Yet Another Feature Request">YAFR</abbr>}
</script>
</div>

[facelessuser]

I've confirmed that this is a bug in abbr, not in the math extensions. I also have a fix.

MATH DEBUG:
f_\text{YAFR}

MATH DEBUG f_\text{YAFR}
<p>Our function <span class="arithmatex"><span class="MathJax_Preview">f_\text{YAFR}</span><script type="math/tex">f_\text{YAFR}</script></span> does some things</p>
<div class="arithmatex">
<div class="MathJax_Preview">
f_\text{YAFR}
</div>
<script type="math/tex; mode=display">
f_\text{YAFR}
</script>
</div>

[facelessuser]

I can replicate this bug without external dependencies:

import markdown
text = """
This <https://this.com/{YAFR}>

*[YAFR]: Yet Another Feature Request
"""
print(markdown.markdown(text, extensions=["abbr"]))

<p>This <a href="https://this.com/{YAFR}">https://this.com/{<abbr title="Yet Another Feature Request">YAFR</abbr>}</a></p>