-
Notifications
You must be signed in to change notification settings - Fork 631
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
moving slow tests to nayduck - bls12381 #12456
base: master
Are you sure you want to change the base?
Conversation
cc @olga24912 Can you also have a look? I couldn't add you to reviewers for whatever reason. |
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## master #12456 +/- ##
==========================================
- Coverage 71.44% 71.43% -0.02%
==========================================
Files 838 838
Lines 169339 169340 +1
Branches 169339 169340 +1
==========================================
- Hits 120985 120965 -20
- Misses 43006 43026 +20
- Partials 5348 5349 +1
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
Hm, locally for me these tests are all <15s each and even in the most recent CI run I picked up the majority of them finish well within 30s which I don't think warrants these getting moved all the way to nayduck. But I otherwise agree these don't need to run locally. How long is super long in your experience? |
@nagisa Yeah I agree those aren't too bad each by itself. I was more concerned with the fact that there is like 30 of them. Now looking at the test run in CI, the majority of time is spent on building and other arbitrary things, not the test execution itself. Up to you, I still think those are slow enough and stable enough to put them in nayduck but I'm happy to abandon it also. Extra argument for nayduck is that those are fuzzers but I'm not sure how relevant that is. |
This is somewhat misleading -- my understanding of |
btw feel free to merge your PR first and I'll see if there is anything left to do here |
The bls fuzzer tests are really slow. This slows down and increases the cost of CI, and is generally annoying. Moving those to nayduck.
The tests are implemented as some crazy macros so I'm not exactly sure what I'm doing is right.
Here are the tests when executed in nayduck - https://nayduck.nearone.org/#/run/705
I also checked that they can be run locally and it does with a small fix:
./scripts/nayduck.py -t nightly/bls12381.txt -l
Due to some features fun I needed to hackily add
prepare
feature to neard. I'm looking for suggestion how to do it better. This is because nayduck actually builds neard with the same features as for the test. The neard binary is not used, the test is then run using cargo e.g.