Unicode characters

[GoogleCodeExporter]

It seems Unicode/UTF-8 is not supported. Or, if it's supposed to, it's not 
working.

Example:
When I use the Euro symbol in a string, save and try to open a file, I get an 
error message:
RangeError: argument 1 accesses an index that is out of range
(http://cdn.coursevector.com/minerva/4.0.4/js/lib/ByteArray.js:9)

Note: The program doesn’t display an error message when I save the file, but 
it produces an invalid file instead.

I'm using Windows 7 and the most recent Firefox/Minerva version.

Original issue reported on code.google.com by andreas....@googlemail.com on 7 Jan 2015 at 3:33

[GoogleCodeExporter]

I've tried the online version. It seems that it doesn't support Chinese.
But the Adobe AIR version works well.

Original comment by qq169312...@gmail.com on 7 Jan 2015 at 4:21

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Yes, I think the problem is with Unicode characters beyond the 11-bit limit. It 
did work for the Registered Trademark symbol, for example.

But I was talking about the Euro symbol. In UTF-8, it would be encoded in three 
bytes like this:
1110xxxx 10xxxxxx 10xxxxxx (binary)
So instead of 20AC you would have E282AC (hexadecimal)

While, for up to 11 bits, a character would be encoded in two bytes, like this:
110xxxxx 10xxxxxx

And yes, you are correct. Chinese lies beyond that 11-bit limit, so it's 
actually a good example. However, there's also the range 2000-2BFF which 
contains many important additional symbols, like for currency or mathematical 
operators.

So it would be nice if you could add this functionality, or if not, at least 
detect the problem and display a warning instead of saving a corrupted file.

About the Adobe AIR version, you mean an old version (before version 4), 
correct? I'll check it out.

Original comment by andreas....@googlemail.com on 7 Jan 2015 at 5:05

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Yes, in the old version this works better. I wonder why it stopped working.

Original comment by andreas....@googlemail.com on 7 Jan 2015 at 5:17

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Yes it's an issue I just noticed yesterday and am working on a fix. The issue 
is that Adobe AIR does the encoding from UTF-8 to binary. JavaScript I have to 
do that manually and am trying to find a good encoder/decoder that supports 
UTF-8 correctly.

Original comment by gmariani405 on 7 Jan 2015 at 10:12

• Changed state: Started
• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Since it's just some bit shifting/filtering/combining, I did that just now 
(used Wikipedia UTF-8 entry as reference) and also tested it a bit. You can use 
it if you want.

Original comment by andreas....@googlemail.com on 7 Jan 2015 at 3:05

• Added labels: ****
• Removed labels: ****

Attachments:

• codecUTF8.js

[GoogleCodeExporter]

I actually referenced the same article and got it working, just need to release 
the update. But your version is much cleaner, gotta swap that out now. Thanks!

Original comment by gmariani405 on 7 Jan 2015 at 4:44

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Seems there is actually a shorter way to do this in Javascript:
http://monsur.hossa.in/2012/07/20/utf-8-in-javascript.html

Not sure if this is exactly what you need, though.

Original comment by andreas....@googlemail.com on 7 Jan 2015 at 6:06

• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Please clear app caches and visit minerva now. Version 4.1 has your fixes for 
utf-8 support

Original comment by gmariani405 on 8 Jan 2015 at 10:44

• Changed state: Fixed
• Added labels: ****
• Removed labels: ****

[GoogleCodeExporter]

Yes, it works now. Thanks!

Original comment by andreas....@googlemail.com on 8 Jan 2015 at 3:14

• Added labels: ****
• Removed labels: ****