Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Feature: distinguish ISO-8859-2 from windows-1250 mojibake #168

Open
rspeer opened this issue Feb 11, 2021 · 1 comment
Open

Feature: distinguish ISO-8859-2 from windows-1250 mojibake #168

rspeer opened this issue Feb 11, 2021 · 1 comment

Comments

@rspeer
Copy link
Owner

rspeer commented Feb 11, 2021 •
edited
Loading

ISO-8859-2 covers many of the same characters as Windows-1250, but unfortunately has the characters in different places.

An awkwardly ambiguous case that I've found is that the text SchlĂźsselwĂśrter gets decoded by ftfy as SchlßsselwÜrter, considering it to be Windows-1250 mojibake, when in fact it was ISO-8859-2 mojibake that should have said Schlüsselwörter. Distinguishing these without additional context would require recognizing the awkward capitalization and the extreme unlikeliness of the sequence "ßss".
@rspeer rspeer changed the title Feature: recognize ISO-8859-2 mojibake Feature: distinguish ISO-8859-2 from windows-1250 mojibake Feb 11, 2021
@rspeer
Copy link
Owner Author

rspeer commented Feb 11, 2021

I previously made this note because I thought we weren't supporting ISO-8859-2 mojibake at all, but we are. This word decodes correctly in the context of other ISO-8859-2 mojibake.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
None yet
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
1 participant
@rspeer