Skip to content

Grammar fixes identified while validating the grammar against the DMD "compilable" test suite#3021

Closed
CyberShadow wants to merge 27 commits intodlang:masterfrom
CyberShadow:grammar2
Closed

Grammar fixes identified while validating the grammar against the DMD "compilable" test suite#3021
CyberShadow wants to merge 27 commits intodlang:masterfrom
CyberShadow:grammar2

Conversation

@CyberShadow
Copy link
Member

I am using the DMD "compilable" test corpus as one of the steps of validating the D tree-sitter grammar, which is based on the official specification that will be adjusted here.

This validation identified files in this test corpus which compile successfully using DMD, but are currently not valid programs as specified by the grammar in the language's official specification.

As most of these are due to errors and omissions in the grammar specification (as opposed to compiler bugs), this pull request will contain the necessary fixes to sync up the grammar with the compiler behavior.

Note that this pull request is based on, and includes, #3019, which at the time of writing is not yet merged, so all commits in #3019 also appear here.

@dlang-bot
Copy link
Contributor

Thanks for your pull request, @CyberShadow!

Bugzilla references

Your PR doesn't reference any Bugzilla issue.

If your PR contains non-trivial changes, please reference a Bugzilla issue or create a manual changelog.

The full set of supported opcodes is probably an implementation
detail, so we should at least include definitions for the syntax used
to lex them.
Their definition is identical, and therefore indistinguishable.
$(D ;) is already a possible FunctionBody (as MissingFunctionBody).
Use the same approach as NestingBlockComment.

NB: Tokens and Token are not used anywhere in the grammar any more,
and are therefore only illustrative.
The definitions are identical, and therefore indistinguishable.
A { } block after a label does always introduce a scope.

They are also syntactically identical, and therefore
indistinguishable.
Function literals cannot have contracts.
- Semantically indicate that the various occurrences of $(D ;) all
  mean the same thing

- Add explicit counterpart to e.g. NonEmptyStatement

- Avoiding mixing leaf and non-leaf nodes makes the grammar more
  structured in general.
- The optional BOM is mentioned in prose, but not specified.

- Describe the shebang line, which was hitherto undocumented.

  The described syntax is what the current DMD implementation does.

- Define that empty source files are valid.
According to DMD, @-attributes may occur wherever storage classes are
parsed (see `Parser.parseStorageClasses`).
…terList

const / shared / etc. are allowed before "...", but not required.
Return-ref parameters don't need to specifically be declared with the
exact token sequence "return ref". For example, "ref return" also
works, as does e.g. "return scope int* a".
This token is explicitly supported by DMD in this context.
These were previously defined as a set of tokens loosely arranged in a
table. Unfortunately, this layout was ambiguous, and if interpreted
literally (or by a program) in the same way as the rest of the
grammar, they would mean that the tokens on the same line actually
have to appear one after another in the source file in the specified
order.

As such, we gave to sacrifice a bit of human-readability to achieve
machine readability. If necessary, the former can be later improved
using CSS or additional macros, such as MULTICOLS.
Addresses the following irregularity in the grammar:

auto i = int.init.max;   // valid
auto j = (int).init;     // valid
auto k = (int).init.max; // invalid

UnaryExpression cannot chain into further PrimaryExpression or
PostfixExpression.
Match ArgumentList and DMD behavior.
@CyberShadow
Copy link
Member Author

I am probably going to close this and submit the fixes above as individual PRs, as the lot is too much to review in one go.

Block contracts are allowed, as long as the last contract is using the
expression syntax.
@CyberShadow
Copy link
Member Author

CyberShadow commented Jul 4, 2021

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants

Comments