underscore conversion for italics broken in pandoc-2.4

[ousia]

pandoc-2.4 converts

—_legibility_—

into

<p>—_legibility_—</p>

pandoc-2.3 parsed it as:

<p>—<em>legibility</em>—</p>

I’m afraid it might be a bug.

[jgm]

My guess is that this is due to commit 9b0bd4e

[jgm]

Note: commit 9b0bd4e made possible the fix to #4635, so simply reverting it breaks that test.

The whole lastStrPos thing is a kludge due to not being able to look backwards in the token stream when parsing. #4635 relies on (and other symbols triggering an update to lastStrPos, but the parser for _ emphasis apparently relied on lastStrPos only being updated by regular non-symbol strings.

It would be nice to find a cleaner overall solution. But a simple fix for this would also be welcome.

[bpj]

Workaround:

—[_legibility_]{}—

outputs as

<p>—<span><em>legibility</em></span>—</p>

Tip: add a dummy class so that you can remove the hack with a filter once the bug gets fixed!

[agusmba]

Another workaround would be to switch the order, wouldn't it?

_—legibility—_

[ousia]

@bpj, another workaround would be to use the <em> tag.

@agusmba, I’m afraid the reply is straightforward:

The glyph for the em-dash for roman and italic fonts isn’t the same (at least, in TeX Gyre Pagella).

[agusmba]

Ah, I didn't zoom all the way in to see the difference.
For a moment I thought you were shouting back!
Sorry for the noise.

Although in the example, shouldn't both em-dash be in italic?

[ousia]

@agusmba, I only intended to show the difference between the roman and italic glyphs (it was the best way it came to my mind).

[craigbarnes]

Another example that seems to trigger this regression is:

_filename_|_filetype_

Before 2.4, it used to render as 2 emphasis spans, but now it renders as:

<p><em>filename</em>|_filetype_</p>

[jgm]

Fixed by edc6510