Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Hack around Hunspell crashes on Supplementary Unicode plane codepoints #528

Merged
merged 2 commits into from
Dec 21, 2022

Conversation

antichris
Copy link
Contributor

@antichris antichris commented Oct 4, 2022

The absolutely ancient Hunspell version that was bundled with the SpellChecker Node bindings did not have a fully functional support for full Unicode at the time, and it crashes when faced with non-BMP (U+10000...) codepoints (those include most emoji, among many other things). So we just replace all of those with spaces here.

This fixes #284, fixes #450 and fixes #511.

A more robust solution would entail updating Hunspell in SpellChecker.

The absolutely ancient Hunspell version that was bundled with the
SpellChecker Node bindings did not have a full Unicode support at the
time and it crashes when faced with non-BMP codepoints. So we just
replace all of those with spaces here.
By replacing the non-BMP codepoints with two spaces.
antichris referenced this pull request Dec 16, 2022
@bartosz-antosik bartosz-antosik merged commit 3cd2746 into bartosz-antosik:master Dec 21, 2022
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
2 participants