-
-
Notifications
You must be signed in to change notification settings - Fork 144
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Speed up tests #977
Speed up tests #977
Commits on Oct 27, 2020
-
Cache _list_all we don't need the latest list
The test does not require the list of flows to be updated, to a single cached version will do fine (this call otherwise would take ~40 seconds).
Configuration menu - View commit details
-
Copy full SHA for 9e3fb88 - Browse repository at this point
Copy the full SHA 9e3fb88View commit details -
Reduce the amount of verified runs
Downloading a run takes a non-significant amount of time (est. 300ms on my current setup). It is unnecessary to compare against all >=100 runs, while a handful should do fine (perhaps even just one should do).
Configuration menu - View commit details
-
Copy full SHA for bfb5304 - Browse repository at this point
Copy the full SHA bfb5304View commit details -
Increase the batch size to avoid more than 2 pages
The batch size required in some pages over 40 pages to be loaded, which increased the workload unnecessarily. These changing preserve pagination tests while lowering the amount of round trips required.
Configuration menu - View commit details
-
Copy full SHA for deddc04 - Browse repository at this point
Copy the full SHA deddc04View commit details -
Mark as test_get_run_trace as skip
Since it is already covered by test_run_and_upload_randomsearch.
Configuration menu - View commit details
-
Copy full SHA for 480cee0 - Browse repository at this point
Copy the full SHA 480cee0View commit details -
Filter on dataset id serverside
Speeds up ~25x, and reduces network traffic.
Configuration menu - View commit details
-
Copy full SHA for 402a2ef - Browse repository at this point
Copy the full SHA 402a2efView commit details -
Reduce the amount of pages loaded
Loading a page takes ~600ms. I don't think testing with 3 pages is any worse than 10. I also think this is an ideal candidate of test that could be split up into (1) testing the url is generated correctly, (2) testing a pre-cached result is parsed correctly and (3) testing the url gives the expected response (the actual integration test).
Configuration menu - View commit details
-
Copy full SHA for fd02a9d - Browse repository at this point
Copy the full SHA fd02a9dView commit details -
Simplify model tested in swapped parameter test
If the test is that swapped parameters work, we don't need a complicated pipeline or dataset.
Configuration menu - View commit details
-
Copy full SHA for a9fd67b - Browse repository at this point
Copy the full SHA a9fd67bView commit details -
Add a cli flag to toggle short/long scenarios
Some tests support both, by checking e.g. only a few runs vs all runs.
Configuration menu - View commit details
-
Copy full SHA for 7904ef6 - Browse repository at this point
Copy the full SHA 7904ef6View commit details -
Configuration menu - View commit details
-
Copy full SHA for db26dc9 - Browse repository at this point
Copy the full SHA db26dc9View commit details -
Configuration menu - View commit details
-
Copy full SHA for bbf76f4 - Browse repository at this point
Copy the full SHA bbf76f4View commit details -
Configuration menu - View commit details
-
Copy full SHA for 6186121 - Browse repository at this point
Copy the full SHA 6186121View commit details
Commits on Oct 28, 2020
-
Check the trace can be retrieved individually
To cover for the skipping of test_get_run_trace
Configuration menu - View commit details
-
Copy full SHA for 63e103b - Browse repository at this point
Copy the full SHA 63e103bView commit details -
Configuration menu - View commit details
-
Copy full SHA for a7e6b73 - Browse repository at this point
Copy the full SHA a7e6b73View commit details -
Configuration menu - View commit details
-
Copy full SHA for e7f6b3d - Browse repository at this point
Copy the full SHA e7f6b3dView commit details
Commits on Oct 29, 2020
-
Configuration menu - View commit details
-
Copy full SHA for d85b77a - Browse repository at this point
Copy the full SHA d85b77aView commit details