Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Extended ASCII characters #412

Closed
InfiniteTabs opened this issue Aug 22, 2020 · 5 comments
Closed

Extended ASCII characters #412

InfiniteTabs opened this issue Aug 22, 2020 · 5 comments
Labels
question Further information is requested

Comments

@InfiniteTabs
Copy link

Question
I'm running into an issue where characters like 'ö' throwing an error, then I noticed this part in the code:

bool get isAscii {
    for (var cu in codeUnits) {
      if (cu > 127) return false;
    }
    return true;
  }

Is there a reason to not include all 255 ASCII characters?

Best regards

@InfiniteTabs InfiniteTabs added the question Further information is requested label Aug 22, 2020
@InfiniteTabs InfiniteTabs changed the title Not supporting extended ASCII characters? Extended ASCII characters Aug 22, 2020
@simc
Copy link
Member

simc commented Aug 22, 2020

Hive supports UTF-8 characters for values and ASCII for keys. The reason is that decoding ASCII characters is A LOT faster in Dart than decoding UTF-8 which improves Box opening times significantly.

Edit: ASCII only has 127 characters. Everything beyond is UTF-8.

@InfiniteTabs
Copy link
Author

Thanks a lot for your reply!

Do you know any good ways of converting UTF-8 to ASCII in dart/flutter? I'm thinking about just converting the key right before reading/writing in Hive, for instance replacing any invalid character with \xx.

Best regards

@simc
Copy link
Member

simc commented Aug 22, 2020

I recommend using the dartx String.chars extension.

@apoleo88
Copy link

I recommend using the dartx String.chars extension.

.chars or .characters converts from string to a list of unicode char. I don't understand how to get to ascii from there.

I am trying to use as a key a list of bytes, converting it somehow to unicode, but I keep bumping in conversion errors or invalid characters.
Can you please provide an example and add it to the documentation?

@InfiniteTabs
Copy link
Author

InfiniteTabs commented Dec 28, 2020

Another solution that allows you to use any character is to use base64-encoding, so both when you read or write you can use the encoded key.

final String encodedKey = base64.encode(utf8.encode(key));
box.put(encodedKey, json);

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
question Further information is requested
Projects
None yet
Development

No branches or pull requests

3 participants