Convert DOCX to Markdown: Ordered List in Markdown keeps increasing across headings

[ThachNgocTran]

Explain the problem

In Microsoft Word (DOCX), when starting a new heading (in this specific case: Heading 4), make an ordered list (or numbered list). Definitely, the starting number is reset expectedly. But when converting from DOCX to Markdown, using Pandoc, the numbered list keeps continuing, i.e. keep increasing across headings! This is unexpected.

Expectation: When a new heading starts, the ordered list should be reset (in Markdown). This is to keep it consistent with the DOCX version.

Reproducibility

• Input file: input.docx
• Output file: output.md
• Command line to convert Input to Output: pandoc --wrap=none --extract-media=./ -f docx -t gfm input.docx -o output.md
• Pandoc Version: pandoc-3.4-windows-x86_64
• OS: Windows 10 x64 (22H2)
• Office Editor: LibreOffice 24.8.1.2

[jgm]

When one creates this sort of file with Word, it uses different numId for the two lists, and pandoc expects that. It seems that LibreOffice uses the same numId. So I'm actually puzzled about why Word doesn't display the lists as all part of one list, as pandoc interprets it.

There may be something I'm not understanding right about how these lists are to be encoded, or maybe there's an issue with LibreOffice.

See also #7895.

[jgm]

OK, I think I see what is happening in this case.

The list with numId 2 is being used for BOTH the enumerated lists and the headings (the headings are "level 0", <w:ilvl w:val="0"/>, and the lists are level 1).

I think that's why the number resets; because Word interprets this as a new sublist of a higher-level list (the headings).

[jgm]

Word doesn't seem to use numId for headings in this way, when headings aren't numbered. (And even when they are, the inner list isn't using the same numId, though maybe this would happen if I entered the document differently.)

[jgm]

Anyway, the fix should involve modifying the code in the docx reader that tracks continuing lists and sets the start number accordingly, and making sure that the number is reset when higher-level list item in the same numId series is encountered.

[ThachNgocTran]

@jgm Thank you for the quick response. Originally I got this issue from Google Document: I exported a Google document to Docx, converted to Markdown with Pandoc. Got the issue. Then I tried to remove irrelevant parts (of the Docx) to make the reproducible file for the bug, using LibreOffice Writer (latest version).

So I guess this issue is not LibreOffice's.

[jgm]

We do have code for list items that does what I say above:
https://github.com/jgm/pandoc/blob/main/src/Text/Pandoc/Readers/Docx.hs#L758-L762

The problem is that we don't have anything similar for headings, which we don't parse with these fields.

The heading element is created here:
https://github.com/jgm/pandoc/blob/main/src/Text/Pandoc/Readers/Docx.hs#L695

Somewhere in the code path leading up to this, we need to modify docxListState. This may require some changes in the types used in Docx parsing.

[jgm]

some ideas in issue10258 branch.

[ThachNgocTran]

@jgm Hope this issue is simple and can get fixed in the next iteration of Pandoc! 😀

My original intention was to export Google Document into Markdown (Github flavor) in order to import to Obsidian. Along the way, there were some minor issues such as: without the flag --wrap=none, text in a table cell can be broken into newline (by Pandoc), making Obsidian's Markdown renderer unable to draw the table correctly (one should use <br/> instead of \n for newline in a table cell). Apart from that, the bug in this Post is the only major issue left.

[jgm]

I think it is now fixed (the fix will be in the next release).

For the table issue you mention, try setting --columns to a very high value when writing markdown.