Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Broken parsing of files with Unicode BOM #443

Closed
NightTsarina opened this issue Oct 20, 2023 · 1 comment
Closed

Broken parsing of files with Unicode BOM #443

NightTsarina opened this issue Oct 20, 2023 · 1 comment

Comments

@NightTsarina
Copy link

NightTsarina commented Oct 20, 2023

I spent some time wondering why a Markdown file with YAML front-matter had stopped parsing correctly, until I realised that po4a (or one of the text handling libraries) is interpreting the BOM as a regular character in the same line as the YAML document separator (---), and so it believes the whole front-matter block is a Markdown paragraph.

This source snippet (BOM char shown as <feff>

<feff>---
title: Quick security recommendations for your devices
post_date: 2023.10.17
author: Security in a Box
published: true
teaser_image: ../../../media/en/blog/polygonal-hand-holding-smartphone.png
teaser: "See whare are few first effective steps one can take to better protect their Android, iOS/iPhone, Windows, Mac and Linux devices"
---

Is extracted as:

#. type: Plain text
#: src/blog/quick-security-recommendations-for-your-devices/index.md:8
msgid ""
"<feff>--- title: Quick security recommendations for your devices post_date: "
"2023.10.17 author: Security in a Box published: true teaser_image: ../../../"
"media/en/blog/polygonal-hand-holding-smartphone.png teaser: \"See whare are "
"few first effective steps one can take to better protect their Android, iOS/"
"iPhone, Windows, Mac and Linux devices\""
msgstr ""

If I remove the BOM, then parsing is fixed:

#. type: Yaml Front Matter Hash Value: author
#: src/blog/quick-security-recommendations-for-your-devices/index.md:1
#, no-wrap
msgid "Security in a Box"
msgstr ""

#. type: Yaml Front Matter Hash Value: teaser
#: src/blog/quick-security-recommendations-for-your-devices/index.md:1
#, no-wrap
msgid "See whare are few first effective steps one can take to better protect their Android, iOS/iPhone, Windows, Mac and Linux devices"
msgstr ""

#. type: Yaml Front Matter Hash Value: teaser_image
#: src/blog/quick-security-recommendations-for-your-devices/index.md:1
#, no-wrap
msgid "../../../media/en/blog/polygonal-hand-holding-smartphone.png"
msgstr ""
@mquinson
Copy link
Owner

mquinson commented Jan 4, 2024

Thanks for reporting. It should now be fixed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants