Undocumented change in YAML metadata parsing (Pandoc 2.2.3)

[cagix]

Pandoc 2.2.3 and 2.2.3.1 read

---
foo: 42
...

as

Pandoc (Meta {unMeta = fromList [("foo",MetaInlines [Str "42"])]}) []

However, in pandoc 2.2.2 and earlier, the same YAML would produce a MetaString instead:

Pandoc (Meta {unMeta = fromList [("foo",MetaString "42")]}) []

Is this change of behaviour intended? Since it may break some Lua filters, it should be documented.

[cagix]

running the Linux binaries under Debian Stretch

[msprev]

Another undocumented change in pandoc 2.2.3 and 2.2.3.1:

---
foo: true
...

now produces

Pandoc (Meta {unMeta = fromList [("foo",MetaInlines [Str "true"])]}) []

but in earlier versions this produced a MetaBool:

Pandoc (Meta {unMeta = fromList [("foo",MetaBool True)]}) []

This has currently broken panzer.

[jgm]

These changes weren't intentional. It appears that my changes to YAML parsing in 2.2.3, which were intended to fix one regression, caused several other ones. Clearly we need more extensive tests!

Have you verified that these cases have the old behavior with version 2.2.2.1?

[jgm]

Actually I don't see why '42' should be parsed as a MetaString. MetaInlines seems better and more consistent here (even though that is a change). However, we should keep parsing of 'true' as a boolean, since we actually have a boolean type constructor in MetaValue.

[msprev]

I've verified the boolean case has the old behaviour with version 2.2.2.1. This version of pandoc does not generate any problems for me.

I agree that '42' should be parsed as a MetaInlines. Note that under 2.2.2.1:

---
foo: 42
...

produces:

Pandoc (Meta {unMeta = fromList [("foo",MetaString "42")]}) []

while:

---
foo: "42"
...

produces:

Pandoc (Meta {unMeta = fromList [("foo",MetaInlines [Str "42"])]}) []

which seems wrong...?

[cagix]

Have you verified that these cases have the old behavior with version 2.2.2.1?

@jgm

• 2.2.2: Pandoc (Meta {unMeta = fromList [("foo",MetaString "42")]}) []
• 2.2.2.1: Pandoc (Meta {unMeta = fromList [("foo",MetaString "42")]}) []
• 2.2.3: Pandoc (Meta {unMeta = fromList [("foo",MetaInlines [Str "42"])]}) []
• 2.2.3.1: Pandoc (Meta {unMeta = fromList [("foo",MetaInlines [Str "42"])]}) []

so this change was introduced in 2.2.3 ...

[cagix]

why '42' should be parsed as a MetaString. MetaInlines seems better and more consistent here (even though that is a change)

Well, in both cases a single number ends up as string somehow: in 2.2.2 directly as MetaString, in 2.2.3 as Str within MetaInlines ...

For me, both interpretations can work (although I'd vote to read/represent a number actually as a number). However, since this new behaviour does break things, it should be documented ...

[cagix]

@msprev confirmed.

in 2.2.2.1 foo: 42 is read as MetaString "42", whereas foo: "42" is read as MetaInlines [Str "42"]).

2.2.3.1 yields for both YAML versions the same representation MetaInlines [Str "42"]).

[jgm]

Since we don't have a numerical type in MetaValue (perhaps that was a mistake), we have to represent numbers as strings, so I'm going with MetaInlines. I document the change in the changelog for 2.2.3.2...

[cagix]

@jgm thanks for the quick fix :)