Use pytest-split to parallelize across 3 runners and speedup CI
#985
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
…y + deprecation warning
janosh
approved these changes
Sep 27, 2024
There was a problem hiding this comment.
thanks a lot @esoteric-ephemera! 👍 i resolved the merge conflicts. looks ready to go
janosh
changed the title
pytest-split to parallelize across 3 runners and speedup CI
utf
added
house-keeping
hrushikesh-s
pushed a commit
to hrushikesh-s/atomate2
that referenced
this pull request
Nov 16, 2024
* move ase tests to separate test run, use pytest-split on rest of tests, 3 runs per linux version * add test durations for split * update test split, run notebook test as separate test, update test time * move jupyter notebook test into ase * tweak some clean_dir to tmp_dir to prevent unncessary test file creation * reduce to 4 splits * go back to 3 splits * try to get better ci timing estimate per @janosh's suggestion * fix test split cmd * revert pytest split change for separate pr * sync with ase branch and add test split logic / times * tweak wf * change pytest split algo, see if 5 splits works better for balancing * change to 3 splits * change gruneisen test for time use - just make phonon maker cheaper * add ase to phonon supported codes, enforce string literal in BasePhononMaker * update timing for gruneisen * try to fix ci test wf * ci test wf * merge conflict fixes, try reorg tests * pcmt, other tweaks * remove lingering ase frechet filter remarks --------- Co-authored-by: Janosh Riebesell <janosh.riebesell@gmail.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Summary
Splits atomate2 tests into batches using
pytest-split(see related pymatgen change a while ago). Test times were generated from a CI run and averaging over python versions. This PR is synced up with #940 per the discussion there. Copying @janoshCurrent timing info
The forcefield Gruneisen workflow previously took a very long time, ~845 sec in CI. Making some small tweaks to its
BasePhononMakerhas reduced the time significantly to about 130 sec. Now the tests run at 310 sec/split for 3 splitsPrevious timing info
The best load balancing is probably 2-3 splits, since one workflow (forcefield Gruneisen) takes a very large amount of time
Some splits vs. timing notes: