Binary Fields

[kevincox]

Right now it seems that everything is assumed to be UTF-8 as it is passed back as a Rust String. Journald is a binary protocol so it has no limitation on UTF-8. Is there a way to access the non-UTF-8 logs (like code dumps)?

[kevincox]

Reading the code it looks like String::from_utf8_lossy is used which will silently corrupt binary data!

rust-journald/src/reader/mod.rs

Line 188 in 237fbe0

let field = String::from_utf8_lossy(b);

[yu-re-ka]

If a field contains non-UTF-8 binary data, does it still contain a UTF-8 encoded field name?
If it's still possible to extract the field name from fields when they contain non-UTF-8 binary data, it would be good to change the type of JournalEntryFields to BTreeMap<String, Vec<u8>> and provide a method to interpret the data as UTF-8 from there. I assume the user will have to know if the data should be interpreted as UTF-8 or something else.

[kevincox]

does it still contain a UTF-8 encoded field name?

All field names are [A-Z0-9]_ only. https://manpages.debian.org/testing/libelogind-dev-doc/sd_journal_print.3.en.html#DESCRIPTION

It would be good to change the type of JournalEntryFields to BTreeMap<String, Vec> and provide a method to interpret the data as UTF-8 from there.

This makes sense to me. You can always convert String to Vec<u8> for insertion and we can provide helper methods to call lossy or return an error if the field isn't UTF-8.

[yu-re-ka]

Please have a look at #13