Slow parsing of HTML tables with loaded cells

[defanor]

Attempting to parse HTML tables akin to those of HN comments takes the time that seems to grow exponentially as the amount of tr elements grows: here it takes about 4 seconds with 9 comments, 8 with 10, 16-18 with 11, etc. Empty tables don't have the same effect, apparently the inner elements contribute to that somehow.

Reproduction:

pandoc -f html -t html <(curl https://news.ycombinator.com/item?id=12274999)

Versions:

pandoc 1.19.2.4
Compiled with pandoc-types 1.17.0.5, texmath 0.9.4.1, skylighting 0.1.1.5

The Glorious Glasgow Haskell Compilation System, version 8.0.1

[defanor]

Originally found while attempting to parse HTML documents (using pandoc as a library), so apparently it's a parsing issue -- not that of conversion or printing.

[jgm]

Interestingly, if you run tidy on this HTML, pandoc parses it almost instantly. So looking at the tidy warnings might give some useful insight into what is causing the problem. It isn't tables per se -- it's probably caused by unclosed tags within table cells, or something like that. Note: I get lots of errors/warnings both with tidy and with the HTML5 validator at https://html5.validator.nu. For example, at line 20, column 670, there's a `<p>` tag inside a `<span>` element -- that's not allowed. No wonder pandoc gets confused. It would still be good if pandoc could handle this kind of broken HTML, though.

[jgm]

If you use `-f html-native_spans`, then it works fine. So that shows that it's getting confused by the unmatched div tags in this document.

[jgm]

Look at line 421: <div class="comment"> <span class="c00">Pandoc just make me happy. Thanks John &amp; contributors for your work!<span> </span><div class='reply'> <p><font size="1"> </font> </div></td></tr> Here we have two open div tags, then a close div tag and a close td. Pandoc should know to close the open div tag when it hits the close td.