Skip to content

[deps][ci] compiling all depsets in single job #57957

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Merged
aslonnie merged 3 commits into from
Oct 23, 2025
Merged

[deps][ci] compiling all depsets in single job #57957

Show file tree
Hide file tree
Changes from all commits
Commits
Show all changes
3 commits
Select commit Hold shift + click to select a range
fb4fd1a
compiling all depsets in single job
elliot-barn Oct 21, 2025
da25408
adding todo comment for build all command
elliot-barn Oct 21, 2025
ba81810
outputing config name in error message for diff
elliot-barn Oct 23, 2025
File filter

Filter by extension

Filter by extension
Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
23 changes: 3 additions & 20 deletions .buildkite/dependencies.rayci.yml
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -19,31 +19,14 @@ steps:
job_env: oss-ci-base_test-py3.11
depends_on: oss-ci-base_test-multipy

- label: ":tapioca: build: raydepsets: compile LLM dependencies"
key: raydepsets_compile_llm_dependencies
- label: ":tapioca: build: raydepsets: compile all dependencies"
key: raydepsets_compile_all_dependencies
Copy link

@cursor cursor bot Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Missing Commands Break Artifact Permissions

The combined depset compilation job is missing the chown -R 2000:100 /artifact-mount and cp ./python/deplocks/llm/* /artifact-mount/ commands. This prevents LLM dependency artifacts from being published to the artifact mount with correct permissions, breaking downstream jobs.

Fix in Cursor Fix in Web

Copy link
Collaborator

@aslonnie aslonnie Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

maybe add the TODO line in the code here?

tags: always
instance_type: small
Copy link

@cursor cursor bot Oct 21, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Bug: Job Downgrade Causes Resource Constraints

The raydepsets_compile_all_dependencies job is set to instance_type: small, but it now includes workloads (like rayimg and multimodal inference dependency compilation) that previously required medium instances. This downgrade may cause resource constraints, leading to job failures or slower execution.

Fix in Cursor Fix in Web

commands:
# TODO: add raydepsets build all command
- bazel run //ci/raydepsets:raydepsets -- build ci/raydepsets/configs/rayllm.depsets.yaml --check
- chown -R 2000:100 /artifact-mount
- cp ./python/deplocks/llm/* /artifact-mount/
job_env: manylinux
depends_on: manylinux

- label: ":tapioca: build: raydepsets: compile ray img dependencies"
key: raydepsets_compile_rayimg_dependencies
tags: always
instance_type: medium
commands:
- bazel run //ci/raydepsets:raydepsets -- build ci/raydepsets/configs/rayimg.depsets.yaml --check
job_env: manylinux
depends_on: manylinux

- label: ":tapioca: build: raydepsets: compile multimodal inference release test dependencies"
key: raydepsets_compile_multimodal_inference_release_test_dependencies
tags: always
instance_type: medium
commands:
- bazel run //ci/raydepsets:raydepsets -- build ci/raydepsets/configs/release_multimodal_inference_benchmarks_tests.depsets.yaml --check
job_env: manylinux
depends_on: manylinux
2 changes: 1 addition & 1 deletion ci/raydepsets/cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -139,7 +139,7 @@ def diff_lock_files(self):
diffs = self.get_diffs()
if len(diffs) > 0:
raise RuntimeError(
"Lock files are not up to date. Please update lock files and push the changes.\n"
f"Lock files are not up to date for config: {self.config_name}. Please update lock files and push the changes.\n"
+ "".join(diffs)
)
click.echo("Lock files are up to date.")
Expand Down
2 changes: 1 addition & 1 deletion ci/raydepsets/tests/test_cli.py
View file
Open in desktop
Original file line number Diff line number Diff line change
Expand Up @@ -655,7 +655,7 @@ def test_diff_lock_files_out_of_date(self):
with self.assertRaises(RuntimeError) as e:
manager.diff_lock_files()
assert (
"Lock files are not up to date. Please update lock files and push the changes."
"Lock files are not up to date for config: test.depsets.yaml. Please update lock files and push the changes."
in str(e.exception)
)
assert "+emoji==2.8.0" in str(e.exception)
Expand Down