build pure-Python wheels without waiting for dependencies #409
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
jameslamb
added
non-breaking
This comment was marked as resolved.
This comment was marked as resolved.
jameslamb
commented
Sep 22, 2025
| package-name: cuopt_sh_client | ||
| package-type: python | ||
| append-cuda-suffix: false | ||
| pure-wheel: true |
Member
Author
There was a problem hiding this comment.
This should fix the failures in the nightly wheel tests:
[rapids-github-run-id] Querying the GitHub API to determine relevant run of 'build.yaml'.
Downloading and decompressing cuopt_wheel_python_cuopt_sh_client from Run ID 17905116634 into /tmp/tmp.hYHfRQWjjq
no artifact matches any of the names or patterns provided
RAPIDS logger » [09/22/25 05:08:14]
┌──────────────────────────────────────────────────────────────────────────────────┐
| rapids-retry: retry 1 of 3 | exit code: (1) -> sleeping for 120 seconds... |
Member
Author
|
Most conda-based Python tests were failing with errors like this:
Saw that across all CUDA versions, Python versions, and CPU architectures. I suspect that the root caused might rapidsai/ucxx#516 And that that'd be fixed once we get new I triggered a new build of |
This was referenced Sep 23, 2025
rgsl888prabhu
approved these changes
Sep 23, 2025
Member
Author
|
/merge |
rapids-bot bot
pushed a commit
that referenced
this pull request
Sep 24, 2025
Replaces #359 (my more-complicated earlier attempt at this) This project runs nightly builds and tests on a cron schedule: https://github.com/NVIDIA/cuopt/blob/36a6a1c0edf42cec2cf07c6be3f16531f33515de/.github/workflows/nightly.yaml#L1-L6 Tests need to wait for builds to finish, and that's currently done with some shell scripts that hit the GitHub API, using a mix of `sleep` and polling. This has sometimes resulted in nightly failures (network errors, timeouts, etc.). This PR proposes reducing the risk of such failures by moving that logic into GitHub Actions configuration directly, specifically: * making `build.yaml` trigger `test.yaml` with the GitHub CLI **only after all package builds and publishing have finished** ## Issue Contributes to #122 ## Notes for Reviewers ### How I tested this I manually triggered this run of the "Trigger Nightly cuOpt Pipeline": https://github.com/NVIDIA/cuopt/actions/runs/17935159871 Which triggered this `build` run: https://github.com/NVIDIA/cuopt/actions/runs/17935161536 Which triggered this `test` run: https://github.com/NVIDIA/cuopt/actions/runs/17936474025 Things look ok to me! The `test` run was triggered until after all the relevant package builds and uploads were done, and BEFORE the docker image builds were done (as intended, to not be delayed waiting on them). There are some test failures from artifact-downloading, like this: ```text [rapids-github-run-id] Querying the GitHub API to determine relevant run of 'build.yaml'. Downloading and decompressing cuopt_wheel_python_cuopt_server_cu12_py312_x86_64 from Run ID 17936253863 into /tmp/tmp.pqrBXIhMlP ``` But I think they'll be fixed by merging #409 And the naming changes for the image builds look good 😁 <img width="317" height="203" alt="image" src="https://github.com/user-attachments/assets/31bac7bd-1c4d-4c31-9ce9-9863778c2e89" /> Authors: - James Lamb (https://github.com/jameslamb) Approvers: - Ramakrishnap (https://github.com/rgsl888prabhu) URL: #408
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Description
Looking closely at the CI dependency graphs while working on #122, I noticed some opportunities to improve the end-to-end time for PR CI and nightly builds.
wheel-build-{libB}jobs only need to wait forwheel-build-{libA}jobs to complete if buildinglibBwheels requireslibA. This updates job dependencies here to follow that rule.wheel-build-cuopt-server: start building immediatelywheel-build-cuopt-sh-client: start building immediatelybuild-images: addswheel-build-cupot-sh-clientto jobs this needs to wait for (branch / nightly builds only)cuopt/ci/docker/Dockerfile
Line 65 in 933d810
This PR also remove some unnecessary builds. This project is currently running 16
wheel-build-cuopt-serverjobs (2 CUDA major x 2 CPU architectures x 4 Python versions) when it only needs 2 (1 per CUDA major version), because it publishespy3-none-anywheels (https://pypi.anaconda.org/rapidsai-wheels-nightly/simple/cuopt-server-cu13).cuopt-serverbuildscuopt-sh-clientin nightly testsIssue
Noticed while working towards #122
Notes for Reviewers
Benefits of these changes
Reduced end-to-end time for PR builds and probably for branch/nightly builds.
Reduced risk of failed or incorrect container-image builds resulting from
build-imagesstarting before there arecuopt-sh-clientpackages available.14 fewer
wheel-build-cuopt-serverjobs per PR 😁Checklist