Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Specify case insensitive matching of token upon definition #12

Open
ktsivkov opened this issue Mar 7, 2024 · 1 comment
Open

Specify case insensitive matching of token upon definition #12

ktsivkov opened this issue Mar 7, 2024 · 1 comment
Labels
enhancement New feature or request

Comments

@ktsivkov
Copy link

ktsivkov commented Mar 7, 2024

First I want to say that I really like the api of this library, it is very easy to use.

What I want to propose:

Would be great if we could specify specific tokens to be matched case insensitive.
Lets say I have a token called WHERE, would be great if it also matched where.
i.e.
Tokenizer.DefineTokens(TOKEN_TYPE, []string{"WHERE"}, true)
or
Tokenizer.DefineInsensitiveTokens(TOKEN_TYPE, []string{"WHERE"})

@bzick
Copy link
Owner

bzick commented Apr 3, 2024

I think i's good idea. I'm looking at how to implement the idea

@bzick bzick added the enhancement New feature or request label Apr 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
enhancement New feature or request
Projects
None yet
Development

No branches or pull requests

2 participants