Matching tags with namespaces.

[ioquatix]

Here is an example of an html block:

irb(#<Utopia::Project::Document:0x00007f7fe730abd8>):022:0> CommonMarker.render_doc("<foo id='blah'/>")
=> 
#<CommonMarker::Node(document): sourcepos={:start_line=>1, :start_column=>1, :end_line=>1, :end_column=>16} children=[#<CommonMarker::Node(html): sourcepos={:start_line=>1, :start_column=>1, :end_line=>1, :end_column=>16}, string_content="<foo id='blah'/>\n">]>

irb(#<Utopia::Project::Document:0x00007f7fe730abd8>):023:0> CommonMarker.render_doc("<ns:foo id='blah'/>")
=> 
#<CommonMarker::Node(document): sourcepos={:start_line=>1, :start_column=>1, :end_line=>1, :end_column=>19} children=[#<CommonMarker::Node(paragraph): sourcepos={:start_line=>1, :start_column=>1, :end_line=>1, :end_column=>19} children=[#<CommonMarker::Node(text): sourcepos={:start_line=>1, :start_column=>1, :end_line=>1, :end_column=>19}, string_content="<ns:foo id='blah'/>">]>]>

[gjtorikian]

As of #184, fetching Node information like this has been removed (it may be added again in the future, but the backend has switched from C to Rust).

[ioquatix]

Thanks for the update, if I'm understanding correctly, it's no longer possible to access the AST?

[ioquatix]

If anyone wants the original behaviour, I forked this gem and it has all the original functionality and more: https://github.com/ioquatix/markly (including support for tag namespaces as an extension to common mark).

[gjtorikian]

it's no longer possible to access the AST?

At this moment, correct. The underlying Rust gem does support AST walking but it hasn't been a priority for me to implement in "C glue," so I used the major version break as an opportunity to remove it.

Out of curiosity, what do you use AST walking for? I have found that wlaking over the resulting HTML has been much easier. Sure, it's a second pass, but there are many more tools to manipulate that data.

[ioquatix]

• I use it to normalise and process markdown (markdown input, markdown output).
• I use it to manipulate documents (remove sections, move sections, update links).
• I use it for richer conversions (adding links to source code, mermaid diagrams).
• I use it to extract summaries (first paragraph, title, image, etc).

I think for some use cases, processing the output HTML can be okay, but I would say that loosing this feature is a pretty big loss for this library, given that a lot of people using the 0.x version could be using that interface. For me, it was definitely one of the innovative features - before I was trying to process markdown after the fact using Kramdown, and it was much more robust to process the AST directly. I really liked the C interface you built with the AST in this gem, and my fork was really just about taking that to the next level in terms of searching and moving around nodes.

[gjtorikian]

I don't know if this is going to improve anyone's world or anything, but: I realized recently that I might need to expose the comrak AST parsing/walking here in Commonmark for another project (kivikakk/comrak#258) that I'm working on. I'll be working on this in the next few months so AST traversing may very well be coming back.

(ccing @joeldrapper for #199)

[ioquatix]

Thanks! Well, as you know I forked this project (https://github.com/ioquatix/markly) and still maintain the C API, since I depend heavily on the AST manipulations in my own project.

What was important to me was rich AST operations, like finding, matching and replacing.