-
Notifications
You must be signed in to change notification settings - Fork 248
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
feat: test error spans and messages #7526
Conversation
3c1568b
to
0857654
Compare
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark 'Test Suite Duration'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.20
.
Benchmark suite | Current: 0857654 | Previous: f0c1c7b | Ratio |
---|---|---|---|
AztecProtocol_aztec-packages_noir-projects_noir-protocol-circuits_crates_types |
73 s |
56 s |
1.30 |
This comment was automatically generated by workflow using github-action-benchmark.
CC: @TomAFrench
(re-requested a review because previously it was a draft, now it's ready) |
Seeing this I can't help but wonder if we should have just started with golden tests all along |
Ah, yes, using golden tests would also work. I guess we'd need to put frontend tests in a way similar to the ones in Another thing is that in these tests one can easily see the code and the errors at once. With golden tests I guess the code will be in one file and the error output will be in another one... which isn't a big deal either, though. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM.
This is moving us closer to a golden-tests style testing which is definitely more ergonomic to write tests for.
Yeah, they'd probably need to be in a different folder to be more discoverable. You should still be able to write the test before the fix though. The program would just error in a different way (presumably) similar to running the frontend test today before the feature is implemented.
No, the code and output would be in the same file (see the examples of that repository I linked) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
⚠️ Performance Alert ⚠️
Possible performance regression was detected for benchmark 'Compilation Time'.
Benchmark result of this commit is worse than the previous benchmark result exceeding threshold 1.20
.
Benchmark suite | Current: 8cb8e0d | Previous: 3869fe9 | Ratio |
---|---|---|---|
private-kernel-tail |
1.218 s |
0.988 s |
1.23 |
This comment was automatically generated by workflow using github-action-benchmark.
CC: @TomAFrench
We can go some part of the way towards this with #7461 (but granted, we still lose full pattern matching) |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
I guess once we add error codes we can include them like "^^^ [E1234] some message" and parse that (another big refactor, but a much simpler one) |
Yep, exactly |
* master: feat: test error spans and messages (#7526)
chore: revert bump of base64ct to `v1.6.0` after `v1.7.0` yanked (noir-lang/noir#7541) feat(performance): Avoid extra Brillig array offsetting for constant indices (noir-lang/noir#7522) feat: test error spans and messages (noir-lang/noir#7526) chore: bump wasm-bindgen (noir-lang/noir#7535) chore: parallelize `test_transform_program_is_idempotent` (noir-lang/noir#7539) chore(cli): Use `noir_artifact_cli::fs` to read artifacts (noir-lang/noir#7391) fix(experimental): Replace most remaining match panics with errors (noir-lang/noir#7536) chore: bump external pinned commits (noir-lang/noir#7537)
Automated pull of development from the [noir](https://github.com/noir-lang/noir) programming language, a dependency of Aztec. BEGIN_COMMIT_OVERRIDE chore!: bump rust edition to 2024 (noir-lang/noir#7533) chore: revert bump of base64ct to `v1.6.0` after `v1.7.0` yanked (noir-lang/noir#7541) feat(performance): Avoid extra Brillig array offsetting for constant indices (noir-lang/noir#7522) feat: test error spans and messages (noir-lang/noir#7526) chore: bump wasm-bindgen (noir-lang/noir#7535) chore: parallelize `test_transform_program_is_idempotent` (noir-lang/noir#7539) chore(cli): Use `noir_artifact_cli::fs` to read artifacts (noir-lang/noir#7391) fix(experimental): Replace most remaining match panics with errors (noir-lang/noir#7536) chore: bump external pinned commits (noir-lang/noir#7537) END_COMMIT_OVERRIDE --------- Co-authored-by: Tom French <tom@tomfren.ch> Co-authored-by: guipublic <47281315+guipublic@users.noreply.github.com> Co-authored-by: Tom French <15848336+TomAFrench@users.noreply.github.com> Co-authored-by: guipublic <guipublic@gmail.com>
Description
Problem
Resolves #7455
Summary
Introduces a
check_errors
helper so we can to be used in frontend tests. Here's an example:The idea is that "^^^" lines mark primary errors and "~~~" mark secondary errors.
Pros:
check_errors
functionCons:
trait Default
was introduced in a test it would conflict with the existingDefault
trait, etc.)Additional Context
I decided to translate all tests here because after I started working on this I wanted to see it through, but also because I believe it would make things easier from now on, and we can also move faster.
As I translated tests I put some "TODO"s about small things to improve. Once this PR merges I'll create a follow-up issue to tackle those TODOs (here I decided not to change the compiler's code except in a couple of places where testing like this was impossible because of small bugs or unpredictability of the error messages).
Documentation
Check one:
PR Checklist
cargo fmt
on default settings.