builtins.toXML can return strings with invalid UTF-8 encoding

[NaN-git]

Describe the bug

Applying builtins.toXML to a string with invalid UTF-8 encoding returns

"<?xml version='1.0' encoding='utf-8'?>\n<expr>\n  <string value=\"[...]\" />\n</expr>\n"

where [...] is the input string with invalid UTF-8 encoding.

Steps To Reproduce

1. Download test file:

wget -O - https://github.com/flenniken/utf8tests/raw/refs/heads/main/utf8tests.bin | head -n 208 > utf8tests.bin

2. Evaluate the following Nix expression, e.g. in nix repl:

builtins.toXML (builtins.readFile ./utf8tests.bin)

3. The output contains the invalid UTF-8 input string.

Expected behavior

Either builtins.readFile or builtins.toXML should fail and a proper error message should be displayed.

Additional context

Related to issue #12060.

Checklist

• checked latest Nix manual (source)
• checked open bug issues and pull requests for possible duplicates

---

Add 👍 to issues you find important.

[edolstra]

Team discussion:

• Agreed that this should be either an error or a warning (if we're afraid of breaking stuff). Not a high priority though since it's a bit garbage-in-garbage-out (i.e. currently it's the user's responsibility to ensure that the input strings are all valid UTF-8).

[nixos-discourse]

This issue has been mentioned on NixOS Discourse. There might be relevant details there:

https://discourse.nixos.org/t/2024-12-18-nix-team-meeting-minutes-204/57602/1