Dynamic generate unit test matrix and add cache option #6043
Conversation
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Codecov ReportAll modified and coverable lines are covered by tests ✅
Additional details and impacted files@@ Coverage Diff @@
## main #6043 +/- ##
=======================================
Coverage 88.65% 88.65%
=======================================
Files 314 314
Lines 65688 65688
=======================================
Hits 58234 58234
Misses 7454 7454
Flags with carried forward coverage won't be shown. Click here to find out more. ☔ View full report in Codecov by Sentry. |
hoxbro
force-pushed
the
infrastructure2
branch
2 times, most recently
from
f459009
to
c9d0164
Compare
hoxbro
changed the title
hoxbro
force-pushed
the
infrastructure2
branch
from
e51c39a
to
c9d0164
Compare
[skip ci]
hoxbro
force-pushed
the
infrastructure2
branch
from
c9d0164
to
2665a26
Compare
hoxbro
force-pushed
the
infrastructure2
branch
4 times, most recently
from
187aa2f
to
9e123a3
Compare
hoxbro
force-pushed
the
infrastructure2
branch
from
9e123a3
to
d3eabb3
Compare
This was referenced Dec 22, 2023
|
This pull request has been automatically locked since there has not been any recent activity after it was closed. Please open a new issue for related bugs. |
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
This PR makes it possible to generate the test matrix based on inputs. The options are default, full, and downstream. The main pain point this is trying to solve is to reduce the load on our CI when a package is released by only running one OS and Python version in the unit test suite. This will need holoviz-dev/holoviz_tasks#31 to be merged and similar changes in our other repos.
Another problem is, as far as I can see, we also have a problem with downstream tests in general, where it would use the cache packages if the main branch had run before a release/dev-release from that day and, therefore, not install the latest package, making the downstream tests a bit inconsequential.
Example of downstream dispatch run (only 1 unit test suite is run):
Inspiration: https://iscinumpy.dev/post/cibuildwheel-2-10-0/#only
Note: In #6042, I wrongly assumed I needed to merge it into the main before I could test dispatch. The problem was missing
inputsin workflow dispatch. I have reverted the change on the main branch and pushed it here for completeness' sake.