-
Notifications
You must be signed in to change notification settings - Fork 6k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Re-enable running tests via IR in external tests in cases where they don't pass due to Hardhat heuristics #12736
Conversation
9a135e8
to
e4b2977
Compare
e4b2977
to
4f6815c
Compare
There are 15 failures for ElementFi via IR unrelated to NomicFoundation/hardhat#2115 (see job 994880):
|
b9bde34
to
c8803e6
Compare
4f6815c
to
4e7ef7c
Compare
Turns out that all 15 failures were caused by #12738. The reverts were caused by external calls to wrongly calculated address and that's why they did not have a revert message. The issue I found in Hardhat is still real but it's a different thing that I just happened to find by accident. |
4e7ef7c
to
cca9c79
Compare
The failure in OZ test seems to be just a case of NomicFoundation/hardhat#2453 triggered under more specific conditions (see NomicFoundation/hardhat#2453 (comment)). @ekpyron's #12744 fixes the failures on ElementFi so I won't be adding workarounds for them in this PR. It does not fix the OZ one though so I'm disabling this test case in OZ. With that, all tests should start passing here once #12744 is merged. |
d36ca43
to
7404526
Compare
11a8c49
to
438c082
Compare
7404526
to
1ddca4d
Compare
Looks like this has a merge conflict now - but apart from that there's nothing blocking this anymore, is there? |
…don't pass due to Hardhat heuristics
1ddca4d
to
e9f3f93
Compare
Yeah, a small conflict with #12764. They both add some new lines at the same spot. Conflict resolved. |
It's ready unless it turns out that some tests are still failing. |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
The failing euler test is unrelated and hopefully fixed by #12768 (which I just merged).
I'll just go ahead and un-draft and merge this. |
Ah, right. Forgot to undraft it. Thanks! And yeah, euler test failure was unrelated, should be fixed by #12768. |
Depends on #12195.Merged.So far only compilation was enabled in IR presets for OpenZeppelin, PRBMath, ElementFi and (after #12197) Gnosis. This is due to NomicFoundation/hardhat#2115, which is still unresolved. Unfortunately this does not give us any benchmarks via IR for these projects. In this PR I'm re-enabling tests and instead disabling individual test cases that do not pass.
The PR also disables one particularly flaky test file in ElementFi. It has one case that fails often and has been reported upstream (delvtech/elf-contracts#240) but also many tests in the file seem to depend on an external service, which has resulted in a mass failure due to network timeout at least once when the service was down.EDIT: This part moved to #12764.