Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Equation identifier using pandoc-eqnos breaks syntax highlighting #346

Closed
victoraraujo01 opened this issue Dec 17, 2018 · 13 comments
Closed
Labels
Markdown dialect Neither GFM, nor extensions currently accepted. Res: Fixed Fix is checked in, but it might be a few weeks until a release.

Comments

@victoraraujo01
Copy link

What is the problem?

Syntax highlighting stops working when using equation identifiers for Pandoc with pandoc-eqnos (https://github.com/tomduck/pandoc-eqnos) package. Example:

What I get:
image

Expected result:
image

How can I reproduce it?

Just write anything on the same line of a $$ ending a math block.

Is there any error message in the console?

No.

@yzhang-gh
Copy link
Owner

The math syntax highlight is actually contributed by the community. This is a bit hard for me.

On the other hand, it is not a normal Markdown syntax (and even not valid in LaTeX).

@yzhang-gh yzhang-gh added Help wanted Looking for help. Markdown dialect Neither GFM, nor extensions currently accepted. labels Dec 18, 2018
@victoraraujo01
Copy link
Author

Ok, I understand this is not normal Markdown Syntax, but it is actually a wide employed pattern in Pandoc to reference parts of a document (see pandoc-crossref, pandoc-fignos or pandoc-tablenos). I've actually used the same notation to reference images and tables and markdown syntax highlighting still works just fine.

Could you give directions to which file should be modified to support this? Maybe I could take a look at it and see if I can help.

@yzhang-gh
Copy link
Owner

yzhang-gh commented Dec 18, 2018

Thanks.

Here is the "entrance"

https://github.com/neilsustc/vscode-markdown/blob/fb5df65ba2aa4ab9d81e94c9b911a968b8fbe1a5/package.json#L300-L306

I suspect you only need to remove the ending $ here

https://github.com/neilsustc/vscode-markdown/blob/fb5df65ba2aa4ab9d81e94c9b911a968b8fbe1a5/syntaxes/math_display.markdown.tmLanguage.json#L13

And some references for TextMate Grammars
https://macromates.com/manual/en/language_grammars
http://www.apeth.com/nonblog/stories/textmatebundle.html


Finally, make sure no other syntax is broken

Test code
<!-- inline math -->

Let's say $x + y = z$, ...

$\alpha$$\$$$\\$$\ $

\\$\\$\$$\alpha$

$\gdef\bar#1{#1^2} \bar{y} + \bar{y}$

$$\$$$

$$
<!-- display math -->

$$ \tag{1} x + y = z $$

$$
\begin{array}{cc}
    a & b \\
    c & d
\end{array}
$$

$$
\alpha%comment
$$

$$
\gdef\bar#1{#1^2} \bar{y} + \bar{y}
$$

<!-- functions -->

$$ \alpha, \hat{x}, \frac{x}{y} $$

<!-- character -->

$$ x \, y \; z $$

<!-- non-valid math environment -->

$20,000

$ x + y = z $

$\ $

@victoraraujo01
Copy link
Author

I just made a quick test by making your suggested fix and it did, indeed, solved my problem. Thanks!

I will do some tests later with the test code you provided.

@victoraraujo01
Copy link
Author

Here's the how the test code is highlighted without removing the $ on the line you mentioned:

image
Image 1

And here's the same code by changing the line to the following:

"end": "(?<![^\\\\]\\\\)(?<![^\\\\]\\$)(\\$\\$)([\\s]*)",

image
Image 2

You can see that line 16 shows a different output. The image below shows that the math block in lines 13-16 is not correctly rendered by the markdown preview pane (although it seems to recognize everything until the ending $$ at line 16 as a math block):

image
Image 3

Pandoc, however, does render it correctly:

image
Image 4

In my opinion, the syntax highlight showed in Image 2 should be the correct one, because the $$ in the beginning of line 16 is not correctly escaped. Since, however, Pandoc supports it, maybe we could change the regular expression to the one shown below in order to support both the case I presented and mantain the expected output for the test code you provided?

"end": "(?<![^\\\\]\\\\)(?<![^\\\\]\\$)(\\$\\$)([\\s]*)(\\{\\#.*\\})?$",

@yzhang-gh
Copy link
Owner

Sorry, I find I cannot remember why there is the line 16. Feel free to remove that line when testing.

And the math rendering of Markdown is provided by markdown-it-katex, which assumes the block math delimiter $$ to be either at the start or end of a line. That is why you get that rendering error.

Could you please open a PR when you feel it is good enough? Thanks.

@linsui
Copy link
Contributor

linsui commented Jan 9, 2019

The line 13 should be removed. The line 16 is just an ordinary test. The ending '$' just try to make the highlight behave the same as the rendering. 😂

@danielhep
Copy link

This is an old issue, but has this been resolved yet? It still seems to be an issue.

@sjerdmann
Copy link

Hi, I'd like this issue to be solved, and I'd be happy to help. Is the test code you gave still up-to-date ? I can try and solve it and open a PR.

@yzhang-gh
Copy link
Owner

Is the test code you gave still up-to-date?

I think so. (Just remove lines 13 and 16.)

@sjerdmann
Copy link

Okay then. So here is the updated test code, for the records:

Test code

<!-- inline math -->

Let's say $x + y = z$, ...

$\alpha$$\$$$\\$$\ $

\\$\\$\$$\alpha$

$\gdef\bar#1{#1^2} \bar{y} + \bar{y}$

$$\$$$

<!-- display math -->

$$
\begin{array}{cc}
    a & b \\
    c & d
\end{array}
$$

$$
\alpha%comment
$$

$$
\gdef\bar#1{#1^2} \bar{y} + \bar{y}
$$

<!-- functions -->

$$ \alpha, \hat{x}, \frac{x}{y} $$

<!-- character -->

$$ x \, y \; z $$

<!-- non-valid math environment -->

$20,000

$ x + y = z $

$\ $

Just note line 11 ($$\$$$) is apparently not supported by pandoc, but it is by the katex preview.
So the code rendering is unchanged, with or without the final $ in here:

  • With the $:
    With the $
  • Without the $:
    Without the $

And, this was the point, adding text after the final $$ of a math block is correctly supported if the $ is removed. I just added "foo" on line 20:

  • With the $:
    wdollar_foo
  • Without the $:
    ndollar_foo

Tell me if there are other things you want me to test.

@yzhang-gh
Copy link
Owner

Thanks. I've deleted the $ as it should be the right solution in theory and also we don't see anything broken.

@yzhang-gh yzhang-gh added Res: Fixed Fix is checked in, but it might be a few weeks until a release. and removed Help wanted Looking for help. labels Sep 20, 2020
@sjerdmann
Copy link

Great, thanks !

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
Markdown dialect Neither GFM, nor extensions currently accepted. Res: Fixed Fix is checked in, but it might be a few weeks until a release.
Projects
None yet
Development

No branches or pull requests

5 participants