Nokogiri 1.17 breaking boilerplate processing

[opoudjis]

Appears to be triggered by smartquote processing done as a traverse

[opoudjis]

Somehow, random text or classes are being injected into the XML document tree, with the result that not all children of traverse are Nokogiri::XML::Node.

I am removing traverse and replacing it with XPath in quotes cleanup, and even then, there is content other than Node when I go through XPath. I strongly suspect this is a bug with the newly released Nokogiri 1.17 (which is already crashing for others), but I am trying this hotfix out first.

[opoudjis]

I've wasted three hours trying to work out why Nokogiri 1.17 is crashing, and I do not have three spare hours to waste. A bug has already been reported in sparklemotion/nokogiri#3359, and I have no idea if that is the issue or not, but I am freezing Nokogiri for Metanorma on 1.16. I will use html2doc to do so, as a dependency of isodoc > metanorma-standoc

The error seems to be happening at <script>Latn</script>, so there may be Nokogiri trying to do special processing with HTML scripts; it is happening not when the boilerplate is processed, but when the boilerplate XML is merged into the main document.

[opoudjis]

@CAMOBAP has investigated, and html2doc is too late in the load chain to constrain the version of Nokogiri.

[opoudjis]

I cannot reproduce the binding of Nokogiri locally, it is still going to the specified 1.16.8 from html2doc. The earliest invocation of Nokogiri is asciimath2unitsml and mathml2asciimath, which are followed immediately by fontist and then the relaton suite. html2doc loads only after these.

The two gems are redundant, and will be eliminated: metanorma/metanorma-standoc#947 . But I'm going to use them for now to try to constrain Nokogiri. I clearly do need to constrain it closer to home, preferably isodoc or metanorma-utils.

[opoudjis]

I'll try constraining it in isodoc first

[opoudjis]

It's not working, and frankly, I don't have the headspace for this any more, especially with @ronaldtse demanding that I do other things. @CAMOBAP please help. I have released isodoc 2.12.4 to constrain Nokogiri, I am rerunning https://github.com/metanorma/metanorma-ietf/actions/runs/12246534137 to keep trying to refresh it, and it keeps being ignored. I understand why we are caching gems, but this is catastrophically bad right now, and this release is going to get worse.

[opoudjis]

Half hour has elapsed, Nokogiri still stuck on 2.12.3. This is not a caching issue, isodoc always gets grabbed immediately on release, and updating asciimath2unitsml is not a real solution, though I will try that now anyway. I am at a loss.

[opoudjis]

I can't update asciimath2unitsml, it's refusing to update ffi.

I give up.

[opoudjis]

At @andrew2net 's suggestion, forcing isodoc dependency via standoc

[opoudjis]

Thank you @andrew2net that seems to have worked. I am going to do an emergency fix of metanorma/bipm-si-brochure#225 tonight, and then rerelease all gems with explicit updated standoc dependencies, to force Nokogiri downgrade.

[ronaldtse]

I can't update asciimath2unitsml, it's refusing to update ffi.

According to this page:

• Deprecate all math gems that Plurimath supersedes plurimath/plurimath#307

The asciimath2unitsml gem has been deprecated and replaced by the plurimath gem. Time to remove it from the stack?

Ping @suleman-uzair for help on how to migrate usage here.

[opoudjis]

The asciimath2unitsml gem has been deprecated and replaced by the plurimath gem. Time to remove it from the stack?

Yes, I have scheduled that in ticket metanorma/metanorma-standoc#947

cc @suleman-uzair