EBNF * does not parse correctly

[padfoot9445]

When I was reworking my parser, I arrived at this sub-grammar:

[Production($"NTTypeAndIdentifierCSVElement: NTFunctionArgDeclModifiersCombined NTType Identifier Comma?")]
[Production($"NTFunctionArgDeclModifiersCombined: NTFunctionArgDeclModifier*")]
...

which fails to parse one of my test-cases. However,

[Production($"NTTypeAndIdentifierCSVElement: NTFunctionArgDeclModifiersCombined? NTType Identifier Comma?")]
[Production($"NTFunctionArgDeclModifiersCombined: NTFunctionArgDeclModifier*")]
...

works fine. It may be another case where I have mis-understood what is permissible in the parser, but at least to my understanding these two grammars should be equivalent.

The full code in both situations is listed below:

https://github.com/padfoot9445/NewExpr/tree/junk/working
https://github.com/padfoot9445/NewExpr/tree/junk/notworking

Note that although in working csly is only at version 3.5.4 (I am reluctant to upgrade because of the ominous "faliure" annotation in github), I did try upgrading in the notworking branch, which is at version 3.7.4.

In particular, SmallLangTest/ParserTest has two test faliures (Test__Function_Definition__Does_Not_Throw and Test__Parse_AllPrograms__Does_Not_Throw) in notworking, and working passes all tests; the only modification between them is the csly version and the change as mentioned above.

Edit: I accidentally did something to my git history. It should be fixed now, and I did reproduce the issue on the new branches.

[b3b00]

@padfoot9445 , will look at it

[b3b00]

@padfoot9445 could you provide a minimal source that does not parse please ? this will help for debug.

[padfoot9445]

@padfoot9445 could you provide a minimal source that does not parse please ? this will help for debug.

I will try to do so, but it will have to be tomorrow before I am back at my computer to verify that anything does or does not parse.

However, I would speculate that if you modify BubbleSort.expa in ExamplePrograms/ by removing everything but the IsSorted function, and remove the body of that function (providing an empty body as in { } (not tested if this works yet), or a dummy expression (e.g. {1;}), it will reproduce the same behavior.

[padfoot9445]

I just realized you may mean a source as in grammar; that I can't do much speculation on unfortunately. If you did mean source code of my fledgling language, then if you modify anything in ExampleProgram with the intention of reusing my unit tests you should run ./regen.ps1 in the root directory of the repository.

[b3b00]

i can't get your solution to compile. many error :

2>SmallLangType.cs(6,238): Error CS0246 : Le nom de type ou d'espace de noms 'BackingNumberType' est introuvable (vous manque-t-il une directive using ou une référence d'assembly ?)

[b3b00]

I've make it compile as an unit test in CSLY solution removing all references that block compilation. you can see it in https://github.com/b3b00/csly/blob/bugfix/574/tests/ParserTests/Issue574/Issue574Parser.cs

a first unit test taking source code from Test__Function_Definition__Does_Not_Throw is perfectly working !
So I really need a compiling solution to check it

[padfoot9445]

I've make it compile as an unit test in CSLY solution removing all references that block compilation. you can see it in https://github.com/b3b00/csly/blob/bugfix/574/tests/ParserTests/Issue574/Issue574Parser.cs

a first unit test taking source code from Test__Function_Definition__Does_Not_Throw is perfectly working !
So I really need a compiling solution to check it

You have my apologies about the solution thing, I completely forgot to provide build instructions. The files that are missing are automatically generated and can be created by running, in the root directory, one of
./build.ps1 --no-format (--no-format is optional)
./regen.ps1 which does not build the solution but does generate the files
python3 Common/build.py ./ ./NewExpr.yaml if the former two do not work. --no-format may be passed. The first argument corresponds to the folder containing the project and the second argument corresponds to the configuration yaml-file, one of which should be present in SmallLang, NewExpr, and SmallLangTest, named appropriately.

EDIT: you will need to create a python venv via

python3 -m venv .env
.env/bin/activate

And install pyYaml:
pip install pyyaml

Regarding the lexer, to my memory that's just a wrapper around the csly lexer to do testing, and any junk in there is a holdover from before I used csly.

[b3b00]

Ok, so I 've somehow succeeded to make it compile and it "successfully" fails on test Test__Function_Definition__Does_Not_Throw with parsing error (i've used contextual error messages for better understanding)

unexpected TypeInt ('int (line 2, column 8)'). Expecting Ref, Readonly, Frozen, Immut, Copy, CloseParen, .
  |
2 |        int y, frozen readonly 
  |        ^^^ expected Ref, Readonly, Frozen, Immut, Copy, CloseParen,

the parsed source code is

collection<[list<[int]>, number]> Transform(copy ref immut readonly frozen number x, 
        int y, frozen readonly 
        immut int z)
        {
            return new collection<[list<[int]>, number]>(x, y: y, z);
            return new int(1);
        }";

So obviously the problem is with the function parameters declaration.
based on this i 've generated a (not so) minimal grammar that reproduces the issue : minimal grammar for issue

I will try to reduce it even more to make it bearable and easily tested.

[b3b00]

an even shorter version : shorter version

I 'm starting to this what can go wrong. Your grammar is perfectly legit but something is wrong in CSLY that forbid parameters wih no modifier (like immut or frozen)

[b3b00]

And for memory the definitive simpler gammar : https://b3b00.github.io/CslyViz/grammar#Q1NMWS1NQVJLI2dlbmVyaWNMZXhlciBUb2tlbjsKCltDdXN0b21JZF0gSWRlbnRpZmllciA6ICJfQS1aYS16IiAiXzAtOUEtWmEteiI7CgoKCltTdWdhcl0gQ29tbWEgOiAiLCI7CgpbS2V5V29yZF0gU3BlY2lmaWVyMTogInNwZWNpZmllcjEiOwoKW0tleVdvcmRdIFNwZWNpZmllcjI6ICJzcGVjaWZpZXIyIjsKCiNbS2V5V29yZF0gSW1tdXQgOiAiaW1tdXQiOwoKCltLZXlXb3JkXSBUeXBlIDogInR5cGUiOwoKI1tLZXlXb3JkXSBUeXBlTnVtYmVyIDogIm51bWJlciI7CgpwYXJzZXIgUGFyc2VyOwoKcm9vdCA6IE5UVHlwZUFuZElkZW50aWZpZXJDU1ZFbGVtZW50KiA7CgotPiBOVFR5cGVBbmRJZGVudGlmaWVyQ1NWRWxlbWVudDogTlRNYW55U3BlY2lmaWNpZXJzIE5UVHlwZSBJZGVudGlmaWVyOwoKTlRNYW55U3BlY2lmaWNpZXJzOiBOVFNwZWNpZmllcio7CgpOVFNwZWNpZmllcjogWyBTcGVjaWZpZXIxIHwgU3BlY2lmaWVyMl0gOwoKTlRUeXBlOiBOVEJhc2VUeXBlOwoKI05UVHlwZUNTVjogTlRUeXBlIChDb21tYSBbZF0gTlRUeXBlKSo7CgpOVEJhc2VUeXBlOiAgVHlwZSA7JCMkIHR5cGUgeA==

[b3b00]

I've come with a fix but I need some more time to make sure there is no issue anywhere else. So far all unit tests are green , that's good sign

[b3b00]

If you want to test, you can link your solution to a clone of CSLY (branch bugfix/574)