Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Lexer produces incorrect output with macros inside of numeric suffixes #4

Open
KubaP opened this issue Jan 1, 2023 · 0 comments
Open
Labels
a-Lexer Relating to the lexer c-Bug Incorrect behaviour (sans crashing or hanging) l-General Relating to any OpenGL/Vulkan

Comments

@KubaP
Copy link
Owner

KubaP commented Jan 1, 2023

glsl-lsp version: 0.0.1

Describe the bug
The lexer currently fails to correctly handle one edge case in regards to numeric suffixes.

The following source string:

#define TEST +5
uint i = 5uTEST;

should result in the following token stream:

(define directive) (ident "uint") (ident "i") (op "=") (num "5u") (ident "TEST") (punc ";")

however, it currently results in:

(define directive) (ident "uint") (ident "i") (op "=") (num "5uTEST") (punc ";")

In accordance with the specification, the suffix of a number should end when it matches a valid suffix. Since the lexer attempts to produce better errors, it treats all alphanumeric characters after a number and until a break as a whole suffix.

Solution
Fixing this would require keeping track of all currently defined macro names, so that if we come across one in a suffix, we produce a number token followed by an ident token, to allow the parser to correctly expand the macro. At the very least, we only need to parse the name of a macro which will be the first token, (if even present). We would also need to parse undefine directives.

@KubaP KubaP added the c-Bug Incorrect behaviour (sans crashing or hanging) label Jan 1, 2023
@KubaP KubaP added this to glsl-lsp Dec 27, 2023
@KubaP KubaP moved this to Todo in glsl-lsp Dec 27, 2023
@KubaP KubaP added a-Lexer Relating to the lexer l-General Relating to any OpenGL/Vulkan labels Jul 5, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
a-Lexer Relating to the lexer c-Bug Incorrect behaviour (sans crashing or hanging) l-General Relating to any OpenGL/Vulkan
Projects
Status: Todo
Development

No branches or pull requests

1 participant