Use of byte index vs. character index

[knu]

It seems the ts and te values are byte index, not character index even if you feed a multibyte string to the parser. It can be hard to have to convert index values around for one to use this parser because you normally parse a regexp as a multibyte text.

cf. rubocop/rubocop#8989

Is there any plan to optionally provide character index in addition to or instead of byte index? Thanks!

[jaynetics]

@knu thank you for pointing this out. ts and te are indeed not very useful. these byte indexes are provided by Ragel which this gem is based on. Ragel doesn't provide the char index, but we could calculate it in Ruby and make it available. i'll look into that shortly.

[jaynetics]

@knu i've just released v2.0.0 where the indices are now character-based

[knu]

@jaynetics That's great news! Thank you so much for your hard work!