You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
First I want to say that I really like the api of this library, it is very easy to use.
What I want to propose:
Would be great if we could specify specific tokens to be matched case insensitive.
Lets say I have a token called WHERE, would be great if it also matched where.
i.e. Tokenizer.DefineTokens(TOKEN_TYPE, []string{"WHERE"}, true)
or Tokenizer.DefineInsensitiveTokens(TOKEN_TYPE, []string{"WHERE"})
The text was updated successfully, but these errors were encountered:
First I want to say that I really like the api of this library, it is very easy to use.
What I want to propose:
Would be great if we could specify specific tokens to be matched case insensitive.
Lets say I have a token called
WHERE
, would be great if it also matchedwhere
.i.e.
Tokenizer.DefineTokens(TOKEN_TYPE, []string{"WHERE"}, true)
or
Tokenizer.DefineInsensitiveTokens(TOKEN_TYPE, []string{"WHERE"})
The text was updated successfully, but these errors were encountered: