Location excerpts miscompute character positions for unicode and greedy ranges

[benodiwal]

Problem

There are two issues with how error locations are displayed:

1. Greedy version_loc

The version location captures the closing parenthesis when it shouldn't.

Example:
File "dune-project", line 1, characters 11-13:
(lang dune è)

Should be characters 11-12 (just è), not 11-13 (which includes the ))

2. Unicode length miscalculation for East Asian characters

East Asian characters are double-width in terminals, but the underline calculation doesn't account for this.

Example:
File "dune-project", line 1, characters 11-21:
(lang dune 中3.16文)

The string 中3.16文 is 6 characters but shows 10 carets because 中 and 文 are double-width.

Related

This was observed while working on #12794

[benodiwal]

Hey @Alizter, I want to propose a solution to this issue (it is solving it to some extent though). When working on #12844 and when implementing the lexer-level approach suggested by @rgrinberg, I came across this.

NOTE: We have handle for first line i.e., (lang dune è) seperately as that is not handled at lexer level.

When non-ASCII characters are encountered in version strings:

1. The lexer catches the first non-ASCII character immediately and reports the error at that exact position
2. The location now only spans the first invalid character (e.g., characters 14-15 instead of 14-16), which:
   - Eliminates the greedy version_loc problem (no longer captures closing parenthesis)
   - Avoids the Unicode length miscalculation issue (doesn't try to underline the entire multi-byte string)

Example:

  File "dune-project", line 2, characters 14-15:
  2 | (using menhir è)
                    ^
  Error: Invalid atom: contains non-ASCII character(s). Atoms must only contain ASCII characters.

  File "dune-project", line 2, characters 14-15:
  2 | (using menhir 中3.16文)
                    ^
  Error: Invalid atom: contains non-ASCII character(s). Atoms must only contain ASCII characters.

The error location is now precise and the display is correct even for East Asian characters.

[benodiwal]

The Error message can be better here ^^^