You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
I've run into a problem where a UTF-8 encoded string is parsed by Simdjson.parse and one of the resulting strings is encoded in ASCII-8BIT. I can reproduce this like so:
# run with ruby --encoding=UTF-8 if UTF-8 isn't your system default.x='{"m":" – "}'# note the non-ASCII character in the valueputsx.encoding# => #<Encoding::UTF-8>y=Simdjson.parse(x)putsy['m'].encoding# => #<Encoding:ASCII-8BIT>
It seems like the encoding of the output strings should remain the same as the encoding of the input string, right? I'm not sure if this is an issue that belongs here or in the main simdjson repository but I appreciate you taking a look either way.
The text was updated successfully, but these errors were encountered:
I've run into a problem where a UTF-8 encoded string is parsed by
Simdjson.parse
and one of the resulting strings is encoded in ASCII-8BIT. I can reproduce this like so:It seems like the encoding of the output strings should remain the same as the encoding of the input string, right? I'm not sure if this is an issue that belongs here or in the main simdjson repository but I appreciate you taking a look either way.
The text was updated successfully, but these errors were encountered: