Lua error w/ Pandoc 2.17: "Error running Lua: pandoc CommonState expected, got pandoc CommonState"

[jjallaire]

I am seeing this error sometimes occur when running pandoc. (but its actually not deterministic, sometimes files succeed and sometimes they fail). I'd like to find a reproducible case for this -- any idea on what sorts of states could trigger this?

Possibly totally unrelated but I'll mention it here in case it gives a clue: I have several other documents that sometimes succeed and sometimes fails with the message pandoc failed. It doesn't appear to be deterministic either. These files all rendered w/o trouble in Pandoc 2.16.2.

[jjallaire]

I should add that these render are using quarto (https://quarto.org) which makes very extensive use of Lua filters (we have dozens and dozens of filters chained together). It's possible the non-determinism exists because there is some sort of memory corruption that doesn't always manifest. I know that there were many changes to the Lua subsystem made in Pandoc 2.17 -- if there are things you can think of that could tickle a memory management problem that would give me something to start with.

Also, if there is a way to run pandoc with enhanced diagnostics (esp. for Lua) I could also try that.

[jgm]

@tarleb will be able to figure out which of the many Lua subsystem changes might be relevant to this. (He'd also know about diagnostics.)

[jgm]

If you can give us a complete example that sometimes gives this error, it would help. (Including all the Lua filters used...ideally, disentangled from quarto and using just pandoc.)

[jgm]

One thing that would be helpful to know is whether any of your filters use pandoc.read.

[jjallaire]

We do indeed use pandoc.Read frequently.

That said, I discovered pandoc --verbose which helped me narrow down which filter exhibited the problem, and ran with only that filter (which doesn't use pandoc.Read). It turned out that the following idiom was causing the crash (code shortened for brevity, our code included a list of every language that Pandoc highlights):

local kHighlightClasses = pandoc.List({
  "abc",
  "actionscript",
  "ada",
  "agda",
  "apache",
  "asn1",
  "asp",
})

function isHighlightClass(class)
  return kHighlightClasses:includes(class)
end

We changed to this and the crash no longer occurred:

local kHighlightClasses = {
  "abc",
  "actionscript",
  "ada",
  "agda",
  "apache",
  "asn1",
  "asp"
}

function isHighlightClass(class)
  for _, v in ipairs (kHighlightClasses) do
    if v == class then
      return true
    end
  end
  return false
end

Here's the actual commit w/ the fix (same as above but has a longer list of classes): quarto-dev/quarto-cli@ee12f24

I noted from the release notes that pandoc List module was re-written in C. Here we were creating a static list and then traversing it repeatedly -- perhaps that gives something to go on?

[jgm]

Great sleuthing, that is a very plausible thought!

[jgm]

I tried reproducing this. I used the following filter, bug.lua:

local kHighlightClasses = pandoc.List({
  "abc",
  "actionscript",
  "ada",
  "agda",
  "apache",
  "asn1",
  "asp",
  "ats",
  "awk",
  "bash",
  "bibtex",
  "boo",
  "c",
  "changelog",
  "clojure",
  "cmake",
  "coffee",
  "coldfusion",
  "comments",
  "commonlisp",
  "cpp",
  "cs",
  "css",
  "curry",
  "d",
  "default",
  "diff",
  "djangotemplate",
  "dockerfile",
  "dot",
  "doxygen",
  "doxygenlua",
  "dtd",
  "eiffel",
  "elixir",
  "elm",
  "email",
  "erlang",
  "fasm",
  "fortranfixed",
  "fortranfree",
  "fsharp",
  "gcc",
  "glsl",
  "gnuassembler",
  "go",
  "graphql",
  "groovy",
  "hamlet",
  "haskell",
  "haxe",
  "html",
  "idris",
  "ini",
  "isocpp",
  "j",
  "java",
  "javadoc",
  "javascript",
  "javascriptreact",
  "json",
  "jsp",
  "julia",
  "kotlin",
  "latex",
  "lex",
  "lilypond",
  "literatecurry",
  "literatehaskell",
  "llvm",
  "lua",
  "m4",
  "makefile",
  "mandoc",
  "markdown",
  "mathematica",
  "matlab",
  "maxima",
  "mediawiki",
  "metafont",
  "mips",
  "modelines",
  "modula2",
  "modula3",
  "monobasic",
  "mustache",
  "nasm",
  "nim",
  "noweb",
  "objectivec",
  "objectivecpp",
  "ocaml",
  "octave",
  "opencl",
  "pascal",
  "perl",
  "php",
  "pike",
  "postscript",
  "povray",
  "powershell",
  "prolog",
  "protobuf",
  "pure",
  "purebasic",
  "python",
  "qml",
  "r",
  "raku",
  "relaxng",
  "relaxngcompact",
  "rest",
  "rhtml",
  "roff",
  "ruby",
  "rust",
  "scala",
  "scheme",
  "sci",
  "sed",
  "sgml",
  "sml",
  "spdxcomments",
  "sql",
  "sqlmysql",
  "sqlpostgresql",
  "stata",
  "swift",
  "tcl",
  "tcsh",
  "texinfo",
  "toml",
  "typescript",
  "verilog",
  "vhdl",
  "xml",
  "xorg",
  "xslt",
  "xul",
  "yacc",
  "yaml",
  "zsh",
})

function isHighlightClass(class)
  return kHighlightClasses:includes(class)
end

function CodeBlock(el)
  if el.classes[1] and isHighlightClass(el.classes[1]) then
    return {}
  end
end

I then ran pandoc -L bug.lua MANUAL.txt ten times. The result:

[INFO] Running filter bug.lua
[INFO] Completed filter bug.lua in 63 ms
[INFO] Running filter bug.lua
pandoc(16989,0x110f00e00) malloc: Incorrect checksum for freed object 0x7fed5e80b600: probably modified after being freed.
Corrupt value: 0x7fed5e32cf40
pandoc(16989,0x110f00e00) malloc: *** set a breakpoint in malloc_error_break to debug
[INFO] Running filter bug.lua
[INFO] Completed filter bug.lua in 66 ms
[INFO] Running filter bug.lua
[INFO] Completed filter bug.lua in 67 ms
[INFO] Running filter bug.lua
[INFO] Completed filter bug.lua in 63 ms
[INFO] Running filter bug.lua
[INFO] Completed filter bug.lua in 63 ms
[INFO] Running filter bug.lua
pandoc(16996,0x1176d2e00) malloc: Incorrect checksum for freed object 0x7fbd6a02b210: probably modified after being freed.
Corrupt value: 0x7fbd6992d0a0
pandoc(16996,0x1176d2e00) malloc: *** set a breakpoint in malloc_error_break to debug
[INFO] Running filter bug.lua
pandoc(16997,0x10a887e00) malloc: Incorrect checksum for freed object 0x7fa57e02a000: probably modified after being freed.
Corrupt value: 0x7fa57d82d0a0
pandoc(16997,0x10a887e00) malloc: *** set a breakpoint in malloc_error_break to debug
[INFO] Running filter bug.lua
[INFO] Completed filter bug.lua in 63 ms
[INFO] Running filter bug.lua
pandoc(16999,0x11a0f1e00) malloc: Incorrect checksum for freed object 0x7fa2e680b600: probably modified after being freed.
Corrupt value: 0x7fa2e612cf40
pandoc(16999,0x11a0f1e00) malloc: *** set a breakpoint in malloc_error_break to debug

Curious that these aren't the same errors you're getting, but it does point to a memory corruption issue.

[jgm]

Even better repro: bug.lua

local hclasses = {
  "abc",
  "actionscript",
  "ada",
  "agda",
  "apache",
  "asn1",
  "asp",
  "ats",
  "awk",
  "bash",
  "bibtex",
  "boo",
  "c",
  "changelog",
  "clojure",
  "cmake",
  "coffee",
  "coldfusion",
  "comments",
  "commonlisp",
  "cpp",
  "cs",
  "css",
  "curry",
  "d",
  "default",
  "diff",
  "djangotemplate",
  "dockerfile",
  "dot",
  "doxygen",
  "doxygenlua",
  "dtd",
  "eiffel",
  "elixir",
  "elm",
  "email",
  "erlang",
  "fasm",
  "fortranfixed",
  "fortranfree",
  "fsharp",
  "gcc",
  "glsl",
  "gnuassembler",
  "go",
  "graphql",
  "groovy",
  "hamlet",
  "haskell",
  "haxe",
  "html",
  "idris",
  "ini",
  "isocpp",
  "j",
  "java",
  "javadoc",
  "javascript",
  "javascriptreact",
  "json",
  "jsp",
  "julia",
  "kotlin",
  "latex",
  "lex",
  "lilypond",
  "literatecurry",
  "literatehaskell",
  "llvm",
  "lua",
  "m4",
  "makefile",
  "mandoc",
  "markdown",
  "mathematica",
  "matlab",
  "maxima",
  "mediawiki",
  "metafont",
  "mips",
  "modelines",
  "modula2",
  "modula3",
  "mnobasic",
  "mustache",
  "nasm",
  "nim",
  "noweb",
  "objectivec",
  "objectivecpp",
  "ocaml",
  "octave",
  "opencl",
  "pascal",
  "perl",
  "php",
  "pike",
  "postscript",
  "povray",
  "powershell",
  "prolog",
  "protobuf",
  "pure",
  "purebasic",
  "python",
  "qml",
  "r",
  "raku",
  "relaxng",
  "relaxngcompact",
  "rest",
  "rhtml",
  "roff",
  "ruby",
  "rust",
  "scala",
  "scheme",
  "sci",
  "sed",
  "sgml",
  "sml",
  "spdxcomments",
  "sql",
  "sqlmysql",
  "sqlpostgresql",
  "stata",
  "swift",
  "tcl",
  "tcsh",
  "texinfo",
  "toml",
  "typescript",
  "verilog",
  "vhdl",
  "xml",
  "xorg",
  "xslt",
  "xul",
  "yacc",
  "yaml",
  "zsh",
}
local kHighlightClasses = pandoc.List(hclasses)

function isHighlightClass(class)
  return kHighlightClasses:includes(class)
end

function Pandoc(el)
  if isHighlightClass("yaml") then
      io.stderr:write(".")
  end
end

Now do

echo "" | pandoc -L bug.lua --verbose

Sometimes it works, sometimes you get a segfault, sometimes the "incorrect checksum for read object" error noted above.
Also tried this with a smaller list (ten items) but was not able to reproduce the issue.

[tarleb]

I guess this counts as reason #‌427 for why not to use C.

Thank you for wiggling this down, it was a quick fix with that information. Sorry for the inconvenience.

I've only opened a PR for now, just in case more bugs in that package emerge.
Didn't want to wait, so I made a quick bugfix release for pandoc-lua-marshal.

[tarleb]

This is the test code I'm using, it triggered the bug reliably:

local lst = List:new()
for i = 1, 1000 do
  lst[#lst + 1] = tostring(i)
end
assert(lst:includes '999')

The cause for the bug was a missing lua_pop in one of the loops, thereby polluting the Lua stack and causing an overflow.

[daniel1noble]

Hi,

I've just started getting what appears to be a similar error with table crossref.lua. This appeared all of a sudden. I'm on Mac OS 13.2.1 (22D68), with Quarto v1.78.0 through VSCode 1.76.2. I think everything is updated (R ver 4.2.3) and it worked just a couple of weeks ago and now fails with:

Error running filter /Applications/quarto/share/filters/crossref/crossref.lua:
table expected, got Inline

I'm using the gt() package for the table I am rendering and am rendering to a word document. I'm not totally clear from reading these threads if there is a resolution as of yet.

Thanks

[jjallaire]

@daniel1noble This is likely a Quarto issue possibly created by an update to Quarto v1.3? Could you post his issue on the Quarto repo and provide us with the source document that this fails with?

cc @cscheid perhaps related to the new AST work?

[cscheid]

This definitely is on our side and unrelated to pandoc. (@jjallaire For 1.4, let's add a check for errors in our pandoc execution that makes it clear that failed pandoc executions are a quarto bug to help users go to the right place.)

[daniel1noble]

Thanks both! I’ve also logged an issues on quarto-dev just in case, but thanks for looking into this and adding some checks.