Proper way to add "alert"/"note"/"hint"/"warning"/... boxes?

[mgeier]

I apologize if this has been discussed before, but I didn't find anything about it in the issues ...

I often see colored boxes in Markdown cells, e.g. in docs/source/examples/Notebook/Notebook Basics.ipynb (rendered by Github/nbviewer).

Those are typically created with raw HTML <div>s like this:

<div class="alert alert-success">
Blah blah blah
</div>

This looks nice and all, but

1. It's kinda tedious to type
2. The colored boxes (but not their content) get lost in translation to LaTeX

Is there a better way to create those boxes?

[Carreau]

Is there a better way to create those boxes?

Hum, not as far as I am aware, we would have to have some king of markdown role like in sphinx/rst.
Hope the common mark folks come up with a syntax for this soon !

[jankatins]

As a workaround you could use a python object which implements _repr_html_ and _repr_latex_:

class Alert():
    def __init__(self, text):
        self.text = text
    def _repr_html_():
        return '<div class="...">text</div>'
    def _repr_latex_():
        return '...'

This would return both html and latex and would therefore be visible in both. But now you need to remove the cell itself (not sure how that works...)

[mgeier]

@Carreau Thanks, it's good to know that I didn't overlook anything.
@JanSchulz Thanks, but no thanks. I'm looking for a proper solution, not for a work-around.

@Carreau wrote:

Hope the common mark folks come up with a syntax for this soon !

I don't think this will happen anytime soon. This was discussed over and over for years, but nothing was added to CommonMark. A few proposals are have been collected in the CommonMark Wiki under "Directives".

What about implementing one of them in The Notebook as a Markdown extension?

Could someone please point me to the code where Markdown cells are parsed?
Probably there is already an extension available for whatever JavaScript library is used for that?

Here are a few more links:

• admonition support for Python-Markdown: https://pythonhosted.org/Markdown/extensions/admonition.html
• proposed admonition support in pandoc: Add Markdown Admonition Support jgm/pandoc#2610

[Carreau]

What about implementing one of them in The Notebook as a Markdown extension?

Well, I doubt we have enough bandwidth to implement our own markdown parser/renderer, in js and then in nbconvert, and then in all the other software that support the notebook format.

Could someone please point me to the code where Markdown cells are parsed?

notebook/static/notebook/js/textcell.js MarkdownCell.prototype.render ~ Line 370

[takluyver]

Additionally, if everyone gives up on the standard and does their own versions, we end up in the same kind of mess that led to the creation of CommonMark in the first place. So I think we should say no to ad-hoc extensions, and encourage people who don't like that to go and improve CommonMark.

[jankatins]

Just putting in a few good words for a pandoc based notebook/nbconvert: pandoc because that's used on the R side with knitr/RMarkdown and because it is (next to github) the kind of "extended standard". knitr/RMarkdown currently (and more so in the future if e.g. better tables are added to pandoc or the above admonition support) have an advantage on the things they can express vs what can be expressed in the notebook md cells. nbconvert can probably easily switch to pandoc, but the browser would need a md2html webservice (e.g. using pypandoc). Whether it would be worth it? The tables feature might be a great addition, as it would mean that more richly styled html tables could be translated into pdfs.

@mgeier If you don't need interactivity, just html and latex/PDF support (and after pandoc gets everything you want), you could try to write admonitions into md cells (or raw cells?), ignore the error in the notebook and then convert the ipymd file to md using nbconvert and then the md file to whatever pandoc supports via pandoc.Or use python-markdown now, if that has a commandline util which works like pandoc.

[mgeier]

@Carreau

I'm not talking about implementing a Markdown parser (or several of them), I'm talking about an extension (or several of them).

Thanks for the pointer, here's a link.
Apparently, there is already some pre-processing being done before marked is used for the actual conversion.

In the next few days (or weeks) I'll have a closer look how my proposed extension could be added there.

@takluyver

The goal of CommonMark, as far as I understand, is to have an unambiguous spec for the most "common" Markdown features. CommonMark allows, probably even encourages, extensions.

There are many proposed extensions (same link as I gave above), most of them will never be part of CommonMark.
In the discussions linked from the wiki page, the CommonMark authors even encourage people to create extensions.
And since extensions like the one i'm suggesting have been discussed for years and not been added to CommonMark, leads me to predict that it ain't gonna happen.

Fun fact 1: marked (and therefore The Notebook) doesn't use CommonMark
Fun fact 2: The Notebook already uses at least 2 extensions (probably more): inline math with $...$ and raw LaTeX blocks that are interpreted as math.
(Just for future reference, the former might be built-in to marked sometime in the future (markedjs/marked#180))

I don't see a fundamental problem in adding another extension, if it keeps people from using HTML-exclusive work-arounds.

And sure, other tools will have to implement those extensions, too, but that's what happens already, see e.g. spatialaudio/nbsphinx#35.

@JanSchulz

Pandoc is circumstantial to this discussion, it's just another parser.

And I don't want to ignore errors in The Notebook. I want this to work correctly in The Notebook and also in all related tools I care about.

[takluyver]

I'm fine with extensions once there is a generic extensible syntax (see this discussion). What I want to avoid is doing ad-hoc extensions where someone picks some symbols and decides to make them mean a special thing, without regard for compatibility or conflicts.

I know we already make some extensions to the markdown the notebook supports. That does not mean it's OK to add more. Those extensions cause enough headaches, but so many notebooks already rely on them that we can't practically get rid of them.

I'm deliberately being hard on this, because I want to see a generic extension point in CommonMark, and that's not going to happen if we carry on doing ad-hoc extensions.

[mgeier]

@takluyver Thanks for the pointers. I've read a large part of the discussions (and there's a lot to read!), and it doesn't look like this extensible syntax is happening anytime soon. Development seems to go quite slowly currently, and the "base" syntax seems to have highest priority, since version 1.0 of the spec is already overdue.

I know we already make some extensions [...] we can't practically get rid of them.

And you shouldn't!
Those extensions are great, and the syntax is really simple and easy to use.
And being able to write equations in a straightforward way is one of the most important features of the Jupyter Notebook!

Anyway, I think the first step forward is to get rid of marked and switch to CommonMark. I've created a new issue about that: #1371.

Once this is done, I think we can continue discussing extensions.

[kmuehlbauer]

We recently changed our documentation workflow to integrate jupyter notebooks within our reST-documents. The reasons for that, we wanted to only document once. The notebooks can be utilized by the user and it is synced within the rendered docs (eg. using sphinx). The transition process went really smooth and we are happy to gain this much from using jupyter notebooks within our project.

However, these things can't be accomplished so far:

• admonitions
• citations

For admonitions I proposed some workaround for the 'nbsphinx' package (which we are using to convert the notebooks) spatialaudio/nbsphinx#46, which has it's faults, but works. Also in this issue here, several workarounds and hacks have been proposed, neither of which is really favourable.

From the users perspective things should just work. If there weren't the inline math available within the notebooks, this would have been a showstopper. And using <div> sections to get alert-boxes is (to some degree) annoying, too.

I understand, that the cleanest solution is to wait for the changes in the standard. But please also consider an interim extension which takes care of this until the changes are incorporated in the standard. Otherwise and this is already under-way, hacks, workarounds, incompatibilities will float around (just my 2c).

[mgeier]

For those who are interested: Since a proper solution will take a long time to come, I've implemented a temporary work-around in nbsphinx that converts <div> blocks with bootstrap warning and info classes to Sphinx boxes: http://nbsphinx.readthedocs.io/en/latest/markdown-cells.html#Info/Warning-Boxes

[mgeier]

@JamiesHQ Why did you close this?
Is there a solution available?

[JamiesHQ]

@gnestor & @mpacer : what are the next steps on this issue? Thanks!

[gnestor]

There is no plan to implement something like this. However, I have seen several issues related to extending our markdown renderer, so I think the best thing to do is create an issue for implementing a general markdown extension interface so that we can start to act on things like this: #2450

[stevengj]

Admonitions !!! foo are a common extension to Markdown, e.g. supported by mkdocs, in pandoc via an extension (jgm/pandoc#2610), and in flexmark via an extension. Julia's markdown docstrings commonly use them for warnings etcetera. Would be nice to have them in notebooks, but I understand that this is more of an upstream issue for you guys.

[choldgraf]

Another thought here - perhaps rather than adopting (or defining) a new markdown syntax for "admonition" blocks, we could use the cell-level metadata for this.

Cell tags could be a way to tag markdown cells with the metadata needed for a renderer to know what "kind" of markdown cell it is. For example, Jupyter Book uses a "hide_input" tag to control whether cells are hidden. One could similarly imagine a "warning" or "information" tag with cells.

A UI for adding / removing tags from cells is now natively in both the Classic notebook UI, as well as JupyterLab (I am not sure about this for Nteract?).

I think the main thing we'd need in this case is not a new syntax in Markdown, but for Jupyter to formally recommend this as a valid approach, and potentially to define a subset of tags that are recognized as the "official" name for a particular thing (e.g. "warning" vs. "warn", etc). In the future, you could imagine front-ends using these tags to render some cells differently from others.

[zommuter]

Nice idea @choldgraf, however what about nested admonitions? The admonition-tag would thus require opening and closing versions I guess

[choldgraf]

@zommuter yep, that's a good point. I think that out-of-the-box it would be most straightforward to do that without supporting nested admonitions :-/

That said, I feel like if tags were supported for something like this, it doesn't prevent some kind of markup support for this as well (which might be the suggested approach for nested admonitions)

[mgeier]

AFAICT, three possibilities have been suggested:

1. The <div class="alert alert-info"> work-around
2. Using cell tags
3. A proper Markdown/CommonMark extension

IMHO, option (3) would be best, but it might still take a really long time until its time comes.

Option (1) is not great, but it somewhat works today.

Option (2) is somewhere in-between. It has fundamental limitations regarding nesting and I'm not sure whether it's worth spending the effort for a feature that's intrinsically flawed (given that a work-around is available and a better solution will be available in the far future).

I think we should hope for option (3) to come some day, and in the meantime try to improve option (1).

Option (1) works well in:

• Classic Notebook (Binder link: https://mybinder.org/v2/gh/spatialaudio/nbsphinx/master?filepath=doc/markdown-cells.ipynb)
• nbsphinx (https://nbsphinx.readthedocs.io/en/latest/markdown-cells.html), even with LaTeX (https://readthedocs.org/projects/nbsphinx/downloads/pdf/latest/#section.3)

It works well on my local JupyterLab, but for some reason it doesn't seem to work on Binder: https://mybinder.org/v2/gh/spatialaudio/nbsphinx/master?urlpath=lab/tree/doc/markdown-cells.ipynb

It also does not work very well in:

• nbconvert

... and I guess by extension in:

• nbviewer (https://nbviewer.jupyter.org/github/spatialaudio/nbsphinx/blob/master/doc/markdown-cells.ipynb)
• Github: https://github.com/spatialaudio/nbsphinx/blob/master/doc/markdown-cells.ipynb

I think the best short-time investment would be to fix nbconvert (and find out what's going wrong in JupyterLab on Binder)!

[vinipsmaker]

Why not just add support for asciidoc at once? They are as concise as markdown, but translate semantics directly to docbook, so they had admonition blocks you ask for since forever. Markdown is a kind of a email-thing, not a proper solution to full-blown documentation.

[psychemedia]

In passing, I created a simple extensions that implemented simple cell tag based styling. It doesnlt immediately apply style when you add a tag - you need to reload the notebook to see the effect: https://github.com/innovationOUtside/nb_extension_tagstyler

[choldgraf]

Just wanted to give a quick update on this and note that we're experimenting with a JupyterLab plugin that would support these kinds of boxes with MyST Markdown (a cross-platform / extensible flavor of markdown that Jupyter Book uses as well). It's called jupyterlab-myst, still very WIP but would love to hear either support or feedback!

• https://github.com/executablebooks/jupyterlab-myst
• binder link to try it out

That's building on top of a MyST markdown parser/renderer in JavaScript, which is also being worked on here:

• https://github.com/executablebooks/mystjs

The hope is that this MyST JS renderer can be re-used across many other platforms / services / etc as well.

[stevengj]

@choldgraf, note that it would be good to have something that works for programmatic output, e.g.

from IPython.display import display, Markdown
display(Markdown('!!! some admonition'))

in addition to Markdown cells; hopefully that is true for your plugin.

[cohml]

+1 for this feature....

Okay, +10000.

[itcarroll]

I think the best short-time investment would be to fix nbconvert (and find out what's going wrong in JupyterLab on Binder)!

@mgeier I believe the problem on nbconvert is resolved in current versions of its default markdown2html filter. See jupyter/nbconvert#1125.