Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Use a DSL for grammar definition, and add next_token #564

Merged
merged 14 commits into from
Aug 23, 2023
Merged

Use a DSL for grammar definition, and add next_token #564

merged 14 commits into from
Aug 23, 2023

Conversation

AntonyBlakey
Copy link
Contributor

No description provided.

@AntonyBlakey AntonyBlakey requested a review from a team as a code owner August 7, 2023 09:53
@changeset-bot
Copy link

changeset-bot bot commented Aug 7, 2023 •
edited
Loading

🦋 Changeset detected

Latest commit: da722fe

The changes in this PR will be included in the next version bump.

This PR includes changesets to release 1 package
Name Type
@nomicfoundation/slang Minor

Not sure what this means? Click here to learn what changesets are.

Click here if you're a maintainer who wants to add another changeset to this PR

@AntonyBlakey AntonyBlakey self-assigned this Aug 7, 2023
@AntonyBlakey AntonyBlakey marked this pull request as draft August 7, 2023 10:02
@AntonyBlakey AntonyBlakey force-pushed the AntonyBlakey/to_rust_types branch 5 times, most recently from ca341c1 to 6671e9d Compare August 10, 2023 10:17
@AntonyBlakey AntonyBlakey marked this pull request as ready for review August 10, 2023 10:18
@AntonyBlakey AntonyBlakey requested review from OmarTawfik and removed request for a team August 10, 2023 10:18
OmarTawfik
OmarTawfik reviewed Aug 11, 2023
View reviewed changes
.vscode/settings.json Outdated Show resolved Hide resolved
Xanewok
Xanewok suggested changes Aug 14, 2023
View reviewed changes
Copy link
Contributor

@Xanewok Xanewok left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Looks great, however it's hard for me to do an in-depth review, since I am not familiar with the codebase from before this PR and the scope of the changes is very big, as it basically overhauls the parsing.

Left some notes, happy to merge it once we bring the changes from #561 back 🙏

crates/solidity/inputs/language/src/lib.rs Outdated Show resolved Hide resolved
crates/solidity/outputs/cargo/tests/src/doc_examples/visitor_api.rs Show resolved Hide resolved
crates/infra/utils/src/codegen/read_write.rs Outdated Show resolved Hide resolved
crates/codegen/parser_runtime/src/templates/language.tera Outdated Show resolved Hide resolved
crates/codegen/parser_runtime/src/support/stream.rs Outdated Show resolved Hide resolved
@OmarTawfik OmarTawfik linked an issue Aug 22, 2023 that may be closed by this pull request
pratt parser panics on unreachable! #563
Closed
@OmarTawfik OmarTawfik mentioned this pull request Aug 22, 2023
Closed
@OmarTawfik OmarTawfik mentioned this pull request Aug 22, 2023
Closed
@AntonyBlakey
Copy link
Contributor Author

Not a blocker, but I think we should keep it up to date with the generated sources, to eliminate confusion.

I did.

@AntonyBlakey AntonyBlakey added this pull request to the merge queue Aug 23, 2023
Merged via the queue into NomicFoundation:main with commit e326a06 Aug 23, 2023
1 check passed
@AntonyBlakey AntonyBlakey deleted the AntonyBlakey/to_rust_types branch August 23, 2023 07:25
Xanewok added a commit to Xanewok/slang that referenced this pull request Aug 23, 2023
@Xanewok
Make LexicalContext consistent with NomicFoundation#564 definition
42330ff
Xanewok added a commit to Xanewok/slang that referenced this pull request Aug 23, 2023
@Xanewok
Make LexicalContext consistent with NomicFoundation#564 definition
71529a0
@OmarTawfik OmarTawfik mentioned this pull request Aug 24, 2023
Merged
github-merge-queue bot pushed a commit that referenced this pull request Aug 24, 2023
@OmarTawfik
migrate scanner tests to parser tests (#577)
0fbfd0f 
They were disabled in #564 .. This enables them and updates the
snapshots.
@Xanewok Xanewok mentioned this pull request Aug 29, 2023
Closed
github-merge-queue bot pushed a commit that referenced this pull request Aug 29, 2023
@Xanewok
Attempt error recovery in the TerminatedBy parsers (#567)
7c6b6c9 
Based on #564

EDIT:
As asked, this was split into supporting PRs (#579, #580, merged now),
doesn't use the Error CST node anymore and only contains terminated-by
recovery.

To support error side-channel in a backtracking scenario,
`Vec<ParseError>` was added to the `Stream` which serves more as a parse
context now. To backtrack, we first record the position of the stream
and how many errors are there; then, we reset the errors and the
position to the first recorded `Marker` struct (basically copying what
`chumsky` does).

This wasn't changed everywhere as `Stream::set_position` in the lexer
will never emit errors (only parsers do), nor will the optional trivia
parsers. I'm happy to polish the lexer interface a bit to accommodate
for this change later, if that's okay. First, I wanted to make sure the
approach is fine and accepted, until we proceed with the DelimitedBy
error recovery.
@OmarTawfik OmarTawfik linked an issue Aug 31, 2023 that may be closed by this pull request
implement Scanner FSM #552
Closed
@github-actions github-actions bot mentioned this pull request Sep 4, 2023
Merged
github-merge-queue bot pushed a commit that referenced this pull request Sep 27, 2023
@github-actions
Bump Slang Version (#590)
3e4b184 
This PR was opened by the [Changesets
release](https://github.com/changesets/action) GitHub action. When
you're ready to do a release, you can merge this and publish to npm
yourself or [setup this action to publish
automatically](https://github.com/changesets/action#with-publishing). If
you're not ready to do a release yet, that's fine, whenever you add more
changesets to main, this PR will be updated.


# Releases
## @nomicfoundation/slang@0.10.0

### Minor Changes

- [#595](#595)
[`1a258c4`](1a258c4)
Thanks [@Xanewok](https://github.com/Xanewok)! - Attempt error recovery
when parsing incomplete lists

- [#564](#564)
[`e326a06`](e326a06)
Thanks [@AntonyBlakey](https://github.com/AntonyBlakey)! - Parsing
operators with missing operands should no longer panic

- [#564](#564)
[`e326a06`](e326a06)
Thanks [@AntonyBlakey](https://github.com/AntonyBlakey)! - Inline parse
rules are no longer exposed to the API.

- [#564](#564)
[`e326a06`](e326a06)
Thanks [@AntonyBlakey](https://github.com/AntonyBlakey)! - Scanners are
no longer available as methods - use next_token instead

- [#564](#564)
[`e326a06`](e326a06)
Thanks [@AntonyBlakey](https://github.com/AntonyBlakey)! - Scanners are
now grouped into contexts to deal with contextual scanning

### Patch Changes

- [#601](#601)
[`cbd2a79`](cbd2a79)
Thanks [@Xanewok](https://github.com/Xanewok)! - Attempt parser error
recovery between bracket delimiters

- [#599](#599)
[`4bbad48`](4bbad48)
Thanks [@Xanewok](https://github.com/Xanewok)! - Use correct versions
for the `revert` and `global` keywords

- [#561](#561)
[`cb6a138`](cb6a138)
Thanks [@Xanewok](https://github.com/Xanewok)! - Add preliminary
documentation for the `solidity_language` Rust package

Co-authored-by: github-actions[bot] <github-actions[bot]@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers

@OmarTawfik OmarTawfik OmarTawfik approved these changes

@Xanewok Xanewok Xanewok requested changes

Assignees

@AntonyBlakey AntonyBlakey

Labels
None yet
Projects
None yet
Milestone
No milestone
3 participants
@AntonyBlakey @Xanewok @OmarTawfik