Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Remove deprecated oniguruma-to-js #112

Closed
slevithan opened this issue Jan 5, 2025 · 0 comments
Closed

Remove deprecated oniguruma-to-js #112

slevithan opened this issue Jan 5, 2025 · 0 comments

Comments

@slevithan
Copy link

slevithan commented Jan 5, 2025

oniguruma-to-js was archived on 2024-11-15, after it was replaced in Shiki by oniguruma-to-es. Should the remaining references in this library be removed?

Note that the changes here should be removed because they are changing the meaning from ASCII-only to Unicode-based (Oniguruma's \d and \w are Unicode-based by default, unlike JS), so these rewrites not only change what the regexes match but might also make the regexes generated by oniguruma-to-es a bit slower to run. The searches also don't account for cases where the patterns are escaped.

I'm guessing this isn't needed anymore?

And running syntaxLowering seems risky since it makes some inaccurate changes (ex: ASCII-only POSIX classes) and doesn't have full understanding of Oniguruma syntax. Maybe you could run a stripped-down version that only removes comments?


Note that, even for comments, following the rules exactly is fairly complex. Maybe syntaxLowering is already handling this well enough, but in case it's helpful, there are details for this in oniguruma-to-es's Supported features list, under:

  • Flags → Extended
  • Pattern modifiers → Group
  • Pattern modifiers → Directive
  • Other → Comment group
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

1 participant