Markdown Macro for CommonMark

[jeremiahpslewis]

Thanks for this wonderful package...I'd been struggling with the stdlib markdown library for over a week until I found this today and it felt like suddenly the issues I couldn't find a way past in terms of incomplete tests and inconsistent parsing were all resolved at once.

One lovely feature of the existing markdown lib, however, is the md"..." string macro which prevents clunky syntax in notebooks like parse(raw"``\epsilon + 1``"). Do you know whether your library could easily accommodate its own string macro, something like md2"..." or even something that overrides the md macro when CommonMark is loaded into the main environment?

[MichaelHatherly]

Thanks for the kind words @jlewis91, much appreciated.

As you've already linked elsewhere (fonsp/Pluto.jl#957) to the JuliaLang/julia issue regarding replacement of Markdown I won't worry to link back to it again; just to say that yes, that issue is still under consideration and literally just requires myself to have sufficient time available to work towards integration, which is very much lacking at the moment 😆 .

What would help towards that goal is for CommonMark to have as many dependant packages as possible, such as Pluto, which would allow us to iron out issues with CommonMark prior to it becoming a stdlib, where it's interface would for the most part be frozen.

Regarding your actual feature request: for the basic case, the following would work:

macro cm_str(str)
    p = Parser()
    return p(str)
end

though you may want to enable! a number of additional extensions to match Markdown behavior better. I'll point out that what this wont allow is the pseudo-interpolation that Markdown has. I am not a fan of the interpolation behavior in it's current form in Markdown and if we were to re-implement it here as an extension we'd need to map out very clearly how we want it to work properly.

[jeremiahpslewis]

Awesome, thanks for the reply. I can imagine there is a bit of a catch-22 situation here...libraries currently using Markdown implicitly rely upon the interpolation offered by the lib, so the absence of this feature limits drop-in adoption, though I very much get your point of view that the current setup should not be preserved in the context of a full refactor.

Looking forward to seeing how this lib develops, feel free to tag me in any beginner level issues that may come up in this project, I'm eager for sensible ways to support core maintainers and participate more in the community.

[MichaelHatherly]

I can imagine there is a bit of a catch-22 situation here...libraries currently using Markdown implicitly rely upon the interpolation offered by the lib, so the absence of this feature limits drop-in adoption

Yeah, pretty much that. We'll have to implement it prior to consideration for stdlib otherwise it doesn't have feature parity with Markdown which I consider a requirement.

Looking forward to seeing how this lib develops, feel free to tag me in any beginner level issues that may come up in this project

Not exactly a beginner level issue, but getting the interpolation syntax working is the only major outstanding thing. If you ever feel like looking into that one the best place to start is look at how the $ math syntax is implemented and adapting that:

CommonMark.jl/src/extensions/math.jl

Lines 53 to 103 in 1acfbef

	#
	# Dollar math
	#
	struct DollarMathRule end
	function parse_block_dollar_math(p::Parser, node::Node)
	if node.t isa Paragraph
	left = match(r"^(\$+)", node.literal)
	left === nothing && return nothing
	right = match(r"(\$+)$", rstrip(node.literal))
	right === nothing && return nothing
	if length(left[1]) == length(right[1]) == 2
	node.literal = strip(c -> isspace(c) || c === '$', node.literal)
	node.t = DisplayMath()
	end
	end
	return nothing
	end
	block_modifier(::DollarMathRule) = Rule(parse_block_dollar_math, 0)
	const reDollarsHere = r"^\$+"
	const reDollars = r"\$+"
	function parse_inline_dollar_math(p::InlineParser, node::Node)
	dollars = match(reDollarsHere, p)
	if dollars === nothing || length(dollars.match) > 1
	return false
	end
	consume(p, dollars)
	after_opener, count = position(p), length(dollars.match)
	while true
	matched = consume(p, match(reDollars, p))
	matched === nothing && break
	if length(matched.match) === count
	before_closer = position(p) - count - 1
	raw = String(bytes(p, after_opener, before_closer))
	child = Node(Math())
	child.literal = strip(replace(raw, r"\s+" => ' '))
	append_child(node, child)
	return true
	end
	end
	# We didn't match a balanced closing sequence.
	seek(p, after_opener)
	append_child(node, text(dollars.match))
	return true
	end
	inline_rule(::DollarMathRule) = Rule(parse_inline_dollar_math, 0, "\$")

[jeremiahpslewis]

Updated code for this feature (not particularly well tested):

# Macro could also be named md2, or something else...

import Pkg; Pkg.add("CommonMark")
using CommonMark

parser = Parser()
enable!(parser, MathRule())
enable!(parser, TableRule())

macro cm_str(str)
    return cm_md(str)
end

function cm_md(str_ex)
    md = parser(IOBuffer(str_ex))
    esc(md)
end

cm"""

# Test text

```math
1 + \epsilon
```

- list a
- list b

More text here
"""

Given the relative utility, I'll focus on the interpolation issue for now.

[MichaelHatherly]

Closed in #16.

[MichaelHatherly]

Very nice talk (https://www.youtube.com/watch?v=AeEmImQ46L8) @jeremiahpslewis. Couldn't agree more with the points you've raised in it, thanks for taking the time to talk about it.

[jeremiahpslewis]

@MichaelHatherly Thank you for the kind words! And, especially, thank you for the great package and very friendly collaboration!!