Upstream changes for pandoc codefence parsing

[dmadisetti]

Description

I opened this issue in pandoc for handling naked attributes as empty attributes in jgm/pandoc#9791

That means Pandoc will consider

```{attr}
//
```

as

```{attr=""}
//
```

Behind the scenes.

As far as I can tell, this won't change quarto behavior- but wanted to raise it here if there are concerns.

[cscheid]

Thanks for letting us know, we'll keep an eye on it.

FYI, Quarto already needs to do a complicated dance around

```{lang}
...
```

because plain Pandoc 3 doesn't recognize that markdown syntax as a code block (and Quarto needs to, because that's our syntax for block-level code cells). Quarto uses a custom reader for that, and that custom reader carefully converts {lang} into an equivalent code block that Pandoc does parse. I believe that custom reader means we won't be affected by the incoming upstream change, whatever it turns out to be.

The relevant lines are:

quarto-cli/src/resources/pandoc/datadir/readqmd.lua

Lines 65 to 113 in 7506cb6

	local function escape_invalid_tags(str)
	local tags = find_invalid_tags(str)
	-- we must now replace the tags in a careful order. Specifically,
	-- we can't replace a key that's a substring of a larger key without
	-- first replacing the larger key.
	--
	-- ie. if we replace {python} before {{python}}, Bad Things Happen.
	-- so we sort the tags by descending size, which suffices
	table.sort(tags, function(a, b) return #b < #a end)
	local replacements = {}
	for _, k in ipairs(tags) do
	local replacement
	local attempts = 1
	repeat
	replacement = random_string(#k)
	attempts = attempts + 1
	until str:find(replacement, 1, true) == nil or attempts == 100
	if attempts == 100 then
	-- luacov: disable
	print("Internal error, could not find safe replacement for "..k.." after 100 tries")
	print("Please file a bug at https://github.com/quarto-dev/quarto-cli")
	os.exit(1)
	-- luacov: enable
	end
	-- replace all lua special pattern characters with their
	-- escaped versions
	local safe_pattern = k:gsub("([%^%$%(%)%%%.%[%]%*%+%-%?])", "%%%1")
	replacements[replacement] = k
	local patterns = {
	"^([ \t\f\v]*```+[ \t\f\v]*)" .. safe_pattern,
	"(\n[ \t\f\v]*```+[ \t\f\v]*)" .. safe_pattern
	}
	str = str:gsub(patterns[1], "%1" .. replacement):gsub(patterns[2], "%1" .. replacement)
	end
	return str, replacements
	end
	local function unescape_invalid_tags(str, tags)
	for replacement, k in pairs(tags) do
	-- replace all lua special replacement characters with their
	-- escaped versions, so that when we restore the behavior,
	-- we don't accidentally create a pattern
	local result = k:gsub("([$%%])", "%%%1")
	str = str:gsub(replacement, result)
	end
	return str
	end

[dmadisetti]

Oh woops, I was looking at escape_engine. I see. I guess escape_engine escapes engine notation, not extract it. Thanks!