Characters corrupted - linux files

[krasnyii]

Opening a .dat file from Wargame Red Dragon (Linux version) - for instance "ZZ_linux.dat" corrupt the characters (replacing them with boxes ▯)
(See image)
If I save the file and load it into the game, the names are corrupted.
If i replace the boxes ▯ with plain letters and load it into the game, the names are (more!) corrupted.

It would be a pleasure to fix this issue, it would offer a way to port mods on linux :)

[RemiGC]

I'm investigating this. Some text entries in Linux are encoded as UTF32 and takes four bytes instead of two. Part of the fix is rather simple.

However implementing it without some ugly hack is harder. I still haven't found a way to know for sure the type of strings uses in a file, without resorting to encoding detection or looking at the file name.

[is-consulting]

I just checked wether the header contains any inforrmation about the encoding we havent understood earlier but it sadly doesnt look like it.

Encoding detection on a plain byte array representing the string is not easy.
A way might be to test the two known encodings and checking the result for known characters like the alphabet. That would introduce problems in non latin files... Good question.

[is-consulting]

Maybe issue #22 can be fixed as well while someone is working on this.