You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
During the last couple of weeks several issues (#1316, #1342, #1323, #1245) were created because the GlobTokenizer couldn't work with identifiers that contained some special characters, like exclamation, at... and so one.
The issues have been fixed on a case by case basis and the characters have been added to the identifier RegEx, which is used in the GlobTokenizer.
The GlobTokenizer should be more resilient against such characters. A proposal has been mentioned in #1316.
The best would be to make the GlobTokenizer resilient against invalid file/path characters. The tokenizer shouldn't be responsible for any file/path character checking, that's the job of the filesystem. It should be independent of the filesystem and not have any filesystem dependent code in it. E.g. currently there's a WindowsRoot token, in order to support this, the tokenizer mainly prevents the use of : in a path on an unix system. The Globber definition has some special characters, *, ?, /, \. Apart from these the tokenizer should allow all characters to be used in an identifier and later on the specific filesystem should decide if it's valid or not.
I'm going to provide a PR with an implementation, so we could further discuss this.
The text was updated successfully, but these errors were encountered:
During the last couple of weeks several issues (#1316, #1342, #1323, #1245) were created because the GlobTokenizer couldn't work with identifiers that contained some special characters, like exclamation, at... and so one.
The issues have been fixed on a case by case basis and the characters have been added to the identifier RegEx, which is used in the GlobTokenizer.
The GlobTokenizer should be more resilient against such characters. A proposal has been mentioned in #1316.
Quote: #1316 (comment)
I'm going to provide a PR with an implementation, so we could further discuss this.
The text was updated successfully, but these errors were encountered: