-
Notifications
You must be signed in to change notification settings - Fork 4
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Implement judicious skipping of CI jobs #94
Comments
pytest-incremental may help us with skipping unneeded Python tests: https://pypi.org/project/pytest-incremental/ It hasn't been updated in a while though, it may need some work. |
Only run tests based on things that have actually changed. For example, if only Python files have changed, we don't need to run the C++ tests. Contributes to rapidsai/build-planning#94
Only run tests based on things that have actually changed. For example, if only Python files have changed, we don't need to run the C++ tests. Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - Bradley Dice (https://github.com/bdice) - Vyas Ramasubramani (https://github.com/vyasr) - Robert Maynard (https://github.com/robertmaynard) URL: #16642
Only run tests based on things that have actually changed. For example, if only Python files have changed, we don't need to run the C++ tests. Contributes to rapidsai/build-planning#94
Contributes to rapidsai/build-planning#94 Depends on rapidsai/shared-workflows#239 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Bradley Dice (https://github.com/bdice) - GALI PREM SAGAR (https://github.com/galipremsagar) URL: #16713
Contributes to rapidsai/build-planning#94 Depends on rapidsai/shared-workflows#239 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) - Vyas Ramasubramani (https://github.com/vyasr) Approvers: - Bradley Dice (https://github.com/bdice) - GALI PREM SAGAR (https://github.com/galipremsagar) URL: #16713
Only run tests based on things that have actually changed. For example, if only Python files have changed, we don't need to run the C++ tests. Contributes to rapidsai/build-planning#94 Depends on rapidsai/shared-workflows#239 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - Robert Maynard (https://github.com/robertmaynard) - Jake Awe (https://github.com/AyodeAwe) URL: #4634
@KyleFromNVIDIA now that we have a common shared workflow and about a month of data on how pruning the jobs works, do we want to roll this feature out to the rest of RAPIDS and then close this issue? |
Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: #6094
Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: #2466
Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: #392
Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: #1695
Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: #635
Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: #226
Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: #296
Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: #489
In some cases we want to run all CI. Is there a good way to bypass this selection behavior? |
Please describe a scenario in which you want to run a CI job even though the relevant files haven't changed. We can look at ways to force it to run. In the meantime, as a workaround, you can add a comment to a relevant file to force CI to run, then remove it again once CI has passed. |
We could create a new label called "Force CI Run". Then the condition could be something like this: if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_cpp || fromJSON(needs.pr-info.outputs.pr-info).labels.*.name == "Force CI Run" Or we could even modify the if: fromJSON(needs.changed-files.outputs.changed_file_groups).test_cpp || needs.changed-files.outputs.force_ci_run If the label is initially not provided, and is added later, we can then trigger a new workflow run with the label applied by: git commit --allow-empty -m 'Re-run Ci'
git push |
My perspective: if the main goal is just to support PRs like rapidsai/pynvjitlink#107 of the form "I just want to see if a CI issue is unrelated to my code changes", I'd prefer just adding a comment to a central file (like a C++ source file) instead of adding complexity to the |
I've opened rapidsai/shared-workflows#249 and rapidsai/cudf#17064 to test out the approach outlined at #94 (comment). |
The above POC works, but if @jameslamb is correct then it may be better to just modify a C++ file with a comment to force the run. @jakirkham WDYT? |
I made the same assumption as James regarding the purpose of the request and come to the same conclusion. I would prefer to discourage using CI as a way to "just run tests", which is one possible use case. If we are specifically testing the behavior of CI (e.g. CI fails but locally tests pass), then adding a trivial change is not a very high barrier. Alternatively, instead of using PR CI you can use either |
@KyleFromNVIDIA what is left for this? It's difficult for me to tell which repos we still haven't made these changes for. |
Looks like cuspatial has not yet been done. I'll go work on that now. I'm not sure if there's anything else. The way to tell is to look at |
Ok, I was hoping you had a list of what remained and it just hadn't made it into this issue yet. I'm not planning to go through all the repos looking for that workflow, I trust your estimation of what's left. |
Contributes to rapidsai/build-planning#94 Authors: - Kyle Edwards (https://github.com/KyleFromNVIDIA) Approvers: - James Lamb (https://github.com/jameslamb) URL: #1479
@KyleFromNVIDIA could you please do one more check like the one you mentioned in #94 (comment), and then close this issue if you think it's done? |
Currently any change to any RAPIDS repo triggers a complete run of the entire build and test suite of CI jobs. This is very expensive, and is often unnecessary. RAPIDS libraries are typically structured such that they have a clear, linear dependency chain between different components. Some examples:
We can reduce the number of unnecessary jobs that we run in CI by more judiciously skipping jobs that are unnecessary. The simplest approach to do this is by simply checking what files have changed. We have previously implemented a form of this in cudf for both cudf.pandas and now for cudf-polars. To do this, I would propose the following steps:
The text was updated successfully, but these errors were encountered: