Backslashes preceded by spaces wrongly doubled when converting from Dokuwiki to reStructedText

[dbitouze]

Explain the problem.
In the output of:

pandoc -f dokuwiki -t rst <<< "<latex>$\sum_{\substack{(i,j) \in I^2 \i \neq j}}$</latex>"

backslashes preceded by spaces are wrongly doubled:

<latex>$\sum_{\substack{(i,j) \\in I^2 \\i \\neq j}}$</latex>

BTW, no such issue when the string is converted from dokuwiki to org:

$ pandoc -f dokuwiki -t org <<< "<latex>$\sum_{\substack{(i,j)\in I^2\i\neq j}}$</latex>"
<latex>$\sum_{\substack{(i,j)\in I^2\i\neq j}}$</latex>

Pandoc version?

• pandoc: v2.18
• OS: Linux Mageia 8

[jgm]

Simpler demonstration of the phenomenon:

% pandoc -f native -t rst
[ Str "a\\x", Space, Str "\\y" ]
a\x \\y

[jgm]

Here's what I see with docutils' rst2html:

a\\x \\y

gets converted to

<p>a\x \y</p>

and

a\x \y

to

<p>ax y</p>

Finally,

a\x \\y

gets converted to

<p>ax \y</p>

So it seems we should be doubling all literal escape characters. I no longer remember why the code does something fancier:

  escapeString' firstChar opts (c:cs) =
    case c of
         _    | c `elemText` "\\`*_|" &&
                (firstChar || null cs) -> '\\':c:escapeString' False opts cs

[jgm]

My guess is that you want it to be single backslash both times, because (I assume) this is a representation of LaTeX math in dokuwiki? So maybe a second issue is that the dokuwiki reader should be taught to recognize it as such. Can you link to official documentation of this syntax?

[dbitouze]

Here's what I see with docutils' rst2html:

In fact, I'm using rst for a Sphinx-doc LaTeX FAQ. And the double backslashes harm.

[dbitouze]

My guess is that you want it to be single backslash both times, because (I assume) this is a representation of LaTeX math in dokuwiki?

Indeed.

So maybe a second issue is that the dokuwiki reader should be taught to recognize it as such.

Would be nice.

Can you link to official documentation of this syntax?

I'm not sure to understand what you want: an official documentation for the Dokuwiki or the reStructedText side?

• About Dokuwiki, there are LaTeX plugins (e.g. this one) and MathJax plugin but, since I'm not the administrator of the Dokuwiki starting LaTeX FAQ, I don't know which are the one(s) used.
• About reStructedText, the official syntax is here.

[jgm]

Yes, for dokuwiki. Sounds like this syntax is from an optional plugin, not part of the core syntax? That may be why it's not supported.

[dbitouze]

Yes, for dokuwiki. Sounds like this syntax is from an optional plugin, not part of the core syntax?

Indeed.

That may be why it's not supported.

I can understand.

But, anyway, why backslashes are treated differently, depending on whether they are preceded by spaces (doubled) or not (not doubled)?

[jgm]

But, anyway, why backslashes are treated differently, depending on whether they are preceded by spaces (doubled) or not (not doubled)?

As noted above, that's an issue with the RST writer. However, it has been fixed in the commit linked above.

[jgm]

If this is a widely used plugin, I'm open to supporting it, because I don't think this syntax would be likely to appear with another meaning.

[dbitouze]

If this is a widely used plugin,

I don't know if the one used in the Dokuwiki site I'm trying to convert to Sphinx-doc is widely used and I must admit it is rather old (last updated on 2011-04-29).

I'm open to supporting it,

Thanks! But see above.

because I don't think this syntax would be likely to appear with another meaning.

Indeed.

[dbitouze]

But, anyway, why backslashes are treated differently, depending on whether they are preceded by spaces (doubled) or not (not doubled)?

As noted above, that's an issue with the RST writer. However, it has been fixed in the commit linked above.

Well, please remove a doubt from my mind: does this fix make the RST writer:

• never double,
• or always double,

the backslashes?

[jgm]

always double (see above)

[dbitouze]

always double (see above)

Sigh... This will be very harmful when converting to RST files that contain TeX commands that always start with a backslash, very often preceded by a space.

[jgm]

As I explained, this requires that the dokuwiki reader recognize the special TeX math contexts in which backslashes behave differently. Currently the reader does not, because it's not part of core dokuwiki syntax.

[dbitouze]

OK, thanks for the clarification.

AFAICS, the only LaTeX Dokuwiki's plugin is LaTeX Plugin, despite its oldness. Considering its syntax, the ideal would be to not double backslashes:

• that are in <latex>…</latex> tags: not a difficult part I guess,
• that are in math introduced by the other delimiters ($…$, $$…$$, \begin{displaymath}…\end{displaymath}, \begin{eqnarray}…\end{eqnarray}, \begin{eqnarray*}…\end{eqnarray*}, \begin{equation}…\end{equation}, \begin{equation*}…\end{equation*}): more difficult as it would be nice to put them in a math RST directive or role but only when they are not in code blocks or snippets.