RST reader and writer should support roles

[nichtich]

Roles are heavily used by RST tools so it would help to support them. At least interpreted text would be useful. Roles in RST are parsed anyway but ignored by now although they map nicely to bracketed_spans in Markdown. This already works:

$ echo '[text]{.role}' | pandoc -f markdown
<p><span class="role">text</span></p>

So this should also work with rst format:

$ echo '`text`:role:' | pandoc -f rst
<p><span class="role">text</span></p>
$ echo ':role:`text`' | pandoc -f rst
<p><span class="role">text</span></p>

And the expected reverse for rst writer:

$ echo '[text]{.role}' | pandoc -f markdown -t rst
:role:`text`

[jgm]

Are you sure you'd want spans with classes to render as interpreted roles?

Suppose you're converting some HTML to RST. Would you want every span with a class to come out as an interpreted role? That would presumably cause the RST document to be unprocessable with regular docutils, unless these roles were specially defined.

[jgm]

And of course the same applies in reverse: if we don't render classes as roles, we probably shouldn't render roles as classes.

An alternative for unknown roles would be to use a key-value attribute like role, rst-role, or data-rst-role.

[mb21]

Maybe RST roles should be treated analogous to the custom-style attribute we have for docx? i.e.

$ echo '[text]{custom-style=myrole}' | pandoc -f markdown -t rst
:myrole:`text`

[nichtich]

Ok this seemed to simple. Attribute name data-rst_role would be most secure to put roles into but custom-style could be more useful. Might something unexpected happed in a rst-to-docx conversion? I Isf there any equivalent to "RST interpreted text roles" and "docx character classes" in other document formats (TEI?, Docbook?...) so it can be described as an abstract feature of documents? If not, we should better keep it separate.

[mb21]

Might something unexpected happed in a rst-to-docx conversion?

The docx writer simply adds those styles if they don't exist in the reference.docx... but maybe there should be a command-line option to strip all key-value attributes with key custom-style. This is already a problem as of now: if you put them into your markdown document (to style your docx output) and also generate an HTML file, they will litter the HTML output (technically rendering it invalid HTML since they are not prefixed with data-).

I Isf there any equivalent to "RST interpreted text roles" and "docx character classes" in other document formats (TEI?, Docbook?...) so it can be described as an abstract feature of documents?

Good point. I guess to some degree this is a philosophical question – whether a certain piece of markup information should go into an element, a class or an attribute. The three mechanisms are all equivalent in the sense that it's a relatively arbitrary choice between them:

<title>text</title>
<p class="title">text</p>
<p type="title">text</p>

I guess the reasoning for having custom-style implemented as an attribute was that it's being used for styling word docs and doesn't really contain semantic information. Though that line is somewhat blurry: usually you want something styles differently because its semantics are different at least on some level. (HTML classes for example are sometimes used semantically, sometimes only for styling.)

So maybe you could tell us a bit more about what people usually use roles for in RST?

P.S. Indeed, the rst2odt.py tool seems to use custom roles just as the pandoc docx writer uses custom-style attributes.

[nthiery]

As 2 cents to fuel the discussion, here is a use case for this feature.

In rst2ipynb, we use pandoc to convert RST files to markdown and then Jupyter notebooks. Our main use case is to convert documents for SageMath which use a bunch of custom roles for other Sphinx-based tool chains.

It would be nice if we could emulate those custom roles with pandoc one way or the other. For example, if the unknown roles were translated in one form or the other in the AST, we could use filters to process them.

Thanks in advance!

[nichtich]

@nthiery could you provide a small code example?

[nthiery]

Hi @nichtich,

This maybe not the greatest example since it's directly taken from the Sage source tree, but it shows typical use of roles like :func:, :ref:, :class:; we also have :wikipedia: or :trac: roles:

https://github.com/sagemath/sage/blob/master/src/doc/en/thematic_tutorials/tutorial-programming-python.rst

For something really short, here is a made up example:

In trac:`13945` we implement the algorithm from :wikipedia:`Schreier–Sims_algorithm` in the :meth:`strong_generating_set` method of :class:`PermutationGroup`.

[nthiery]

In the above examples, we would want to be able to provide hooks to customize the expansion of the trac and wikipedia into actual URL's. And plausibly also expand the other roles; this should be doable for us using Sphinx's crossref databases of Python and Sage, .

[nichtich]

Revisiting this question I think RST roles should be mapped to and from special attributes or class names like suggested by #3407 (comment). This would add another "special" attribute or class name-prefix in addition to custom-style. I'd prefer the attribute name data-role. It is already used by jQuery mobile but I see no conflict. For instance

In :trac:`13945` we implement the algorithm from :wikipedia:`Schreier–Sims_algorithm`
in the :meth:`strong_generating_set` method of :class:`PermutationGroup`.

could be expressed in Pandoc Markdown with

In [13945]{data-role=trac} we implement the algorithm from
[Schreier–Sims\_algorithm]{data-role=wikipedia} in the
[strong\_generating\_set]{data-role=meth} method of
[PermutationGroup]{data-role=class}.

Please note that there is a default role and some role names are mapped to other kind of markup. Try to convert this with Pandoc:

.. default-role:: strong

`text` equals **text** equals :strong:`text`

.. default-role:: literal

`text` equals ``text`` equals :literal:`text`

.. default-role:: subscript

`text` equals :subscript:`text`

.. default-role:: custom

`text` equals :custom:`text`

If RST roles are mapped to data-role attribute, should Pandoc also map predefined roles to its markup equivalent?

[text]{data-role=strong} equals **text** 

[h4ckninja]

New user here. I came here in search of the ability to extend pandoc to add custom directives and roles for my rST documents. From the discussion, it looks like either can't really be possible right now. Am I understanding this correctly? I'd really like to add in a "me too please" on this.