perf(regular_expression): Remove u16 reader, mark surrogate pairs instead

[leaysgur]

Currently, in non-unicode mode, the pattern string is handled through encode_utf16().

However,

• The parsing process itself is performed only on BMP characters
• Presence of surrogate pairs appears in the AST is limited
• Find the corresponding Span offset is also complicated
  • Besides, there is no corresponding position for each surrogate pair...

From the above, can't we unify the Reader in unicode mode?
Based on the hypothesis, I'd like to try to find out.

[graphite-app]

Your org has enabled the Graphite merge queue for merging into main

Add the label “merge” to the PR and Graphite will automatically add it to the merge queue when it’s ready to merge. Or use the label “hotfix” to add to the merge queue as a hot fix.

You must have a Graphite account and log in to Graphite in order to use the merge queue. Sign up using this link.

[Boshen]

We don't have a codspeed benchmark for it. Let me enable regex parsing in the benchmark.

[codspeed-hq]

CodSpeed Performance Report

Merging #5210 will not alter performance

_{Comparing perf-regexp (b75b388) with main (2fbc283)}

Summary

✅ 29 untouched benchmarks

[leaysgur]

Oh, really? Thank you! 🙈

Although, I was thinking of withdrawing this PR.

Not using encode_utf16() means we never iterate over the units that consists surrogate pairs in the first place.
This would result in ignoring certain specification syntax.

As a result, while test262 is currently passing (though it might just be an oversight), if there isn't a significant improvement, I think it would be better to keep behavior more spec like.

[Boshen]

Oh, really? Thank you! 🙈

Although, I was thinking of withdrawing this PR.

Not using encode_utf16() means we never iterate over the units that consists surrogate pairs in the first place. This would result in ignoring certain specification syntax.

As a result, while test262 is currently passing (though it might just be an oversight), if there isn't a significant improvement, I think it would be better to keep behavior more spec like.

Although, I was thinking of withdrawing this PR.

I don't know about the details, feel free to make your own judgements.