[ci] Use env var to control whether to use S3 bucket in CI #13634

khluu · 2025-02-20T23:14:23Z

Revert all the changes that change model name to S3 and add load_format
^ move all of that to 1 place in arg_utils where if the model is in S3 allowlist and env var is on, model name is changed to S3 and load format is changed to Run AI
Fix a bunch of tests
Some lint fixes

Signed-off-by: kevin <kevin@anyscale.com>

Signed-off-by: <>

github-actions · 2025-02-20T23:14:35Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

Signed-off-by: <>

youkaichao · 2025-02-21T03:39:15Z

vllm/utils.py

@@ -2268,3 +2268,73 @@ def wrapped_init(self, *args, **kwargs) -> None:

    type.__setattr__(cls, '__init__', wrapped_init)
    return cls
+
+
+MODELS_ON_S3 = [


move them to vllm/test_utils.py ?

youkaichao · 2025-02-21T03:40:22Z

vllm/engine/arg_utils.py

@@ -1124,6 +1125,10 @@ def create_engine_config(self,
            f", but got {self.cpu_offload_gb}")

        device_config = DeviceConfig(device=self.device)
+        if envs.VLLM_CI_USE_S3 and self.model in MODELS_ON_S3:


the change looks good to me.

youkaichao · 2025-02-21T03:41:53Z

.buildkite/test-pipeline.yaml

@@ -53,13 +53,13 @@ steps:
  - tests/standalone_tests/lazy_imports.py
  commands:
  - python3 standalone_tests/lazy_imports.py
-  - pytest -v -s mq_llm_engine # MQLLMEngine
-  - pytest -v -s async_engine # AsyncLLMEngine
+  - VLLM_CI_USE_S3=1 pytest -v -s mq_llm_engine # MQLLMEngine


please remove the change before merge.

youkaichao

approve to unblock merge. please fix comments before merge.

Signed-off-by: <>

mergify · 2025-02-21T08:55:21Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @khluu.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: <>

khluu · 2025-02-22T08:18:37Z

pre-commit error also happens on main

mergify · 2025-02-22T08:32:00Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @khluu.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

ywang96

Left two comments - PTAL

tests/spec_decode/test_multi_step_worker.py

vllm/engine/arg_utils.py

Signed-off-by: <>

DarkLight1337 · 2025-02-22T14:08:19Z

~~Hmm why did the CI get skipped?~~

Signed-off-by: <>

khluu and others added 3 commits February 20, 2025 18:46

p

ab41c14

Signed-off-by: kevin <kevin@anyscale.com>

p

dad6fd1

p

a3107e8

Signed-off-by: <>

mergify bot added ci/build structured-output speculative-decoding labels Feb 20, 2025

EC2 Default User added 2 commits February 21, 2025 00:01

p

b3d677b

Signed-off-by: <>

p

c2ac164

Signed-off-by: <>

youkaichao reviewed Feb 21, 2025

View reviewed changes

youkaichao approved these changes Feb 21, 2025

View reviewed changes

p

bf4c1a6

Signed-off-by: <>

mergify bot added the needs-rebase label Feb 21, 2025

EC2 Default User added 4 commits February 21, 2025 21:07

p

8c506d7

Signed-off-by: <>

p

afa6913

Signed-off-by: <>

merge

3e17315

Signed-off-by: <>

p

714da70

Signed-off-by: <>

mergify bot removed the needs-rebase label Feb 22, 2025

EC2 Default User added 5 commits February 22, 2025 02:31

p

a1f25c2

Signed-off-by: <>

p

b1b7b61

Signed-off-by: <>

p

64e4464

Signed-off-by: <>

Merge branch 'main' into khluu/control_s3

acd09b9

p

bf167c3

khluu marked this pull request as ready for review February 22, 2025 08:18

khluu requested review from njhill, LiuXiaoxuanPKU and comaniac as code owners February 22, 2025 08:18

khluu requested review from KuntaiDu, DarkLight1337, ywang96, robertgshaw2-redhat, simon-mo, WoosukKwon and alexm-redhat as code owners February 22, 2025 08:18

mergify bot added frontend v1 labels Feb 22, 2025

khluu added the ready ONLY add when PR is ready to merge/full CI is needed label Feb 22, 2025

DarkLight1337 mentioned this pull request Feb 22, 2025

[CI/Build] Fix pre-commit errors #13696

Merged

mergify bot added the needs-rebase label Feb 22, 2025

ywang96 reviewed Feb 22, 2025

View reviewed changes

tests/spec_decode/test_multi_step_worker.py Outdated Show resolved Hide resolved

vllm/engine/arg_utils.py Show resolved Hide resolved

merge

c2f67dd

Signed-off-by: <>

mergify bot removed the needs-rebase label Feb 22, 2025

EC2 Default User added 2 commits February 22, 2025 13:32

p

848e6c6

Signed-off-by: <>

lint fix

a4b71c4

Signed-off-by: <>

khluu removed request for LiuXiaoxuanPKU, njhill, WoosukKwon and alexm-redhat February 22, 2025 13:41

DarkLight1337 mentioned this pull request Feb 22, 2025

Add pipeline parallel support to TransformersModel #12832

Open

p

fcf7b47

Signed-off-by: <>

DarkLight1337 enabled auto-merge (squash) February 23, 2025 03:19

simon-mo merged commit 2c5e637 into main Feb 23, 2025
58 of 61 checks passed

simon-mo deleted the khluu/control_s3 branch February 23, 2025 03:19

Provide feedback

Saved searches

Use saved searches to filter your results more quickly

[ci] Use env var to control whether to use S3 bucket in CI #13634

[ci] Use env var to control whether to use S3 bucket in CI #13634

khluu commented Feb 20, 2025 •

edited by github-actions bot

Loading

github-actions bot commented Feb 20, 2025

youkaichao Feb 21, 2025

youkaichao Feb 21, 2025

youkaichao Feb 21, 2025

youkaichao left a comment

mergify bot commented Feb 21, 2025

khluu commented Feb 22, 2025

mergify bot commented Feb 22, 2025

ywang96 left a comment

DarkLight1337 commented Feb 22, 2025 •

edited

Loading

[ci] Use env var to control whether to use S3 bucket in CI #13634

[ci] Use env var to control whether to use S3 bucket in CI #13634

Conversation

khluu commented Feb 20, 2025 • edited by github-actions bot Loading

github-actions bot commented Feb 20, 2025

youkaichao Feb 21, 2025

Choose a reason for hiding this comment

youkaichao Feb 21, 2025

Choose a reason for hiding this comment

youkaichao Feb 21, 2025

Choose a reason for hiding this comment

youkaichao left a comment

Choose a reason for hiding this comment

mergify bot commented Feb 21, 2025

khluu commented Feb 22, 2025

mergify bot commented Feb 22, 2025

ywang96 left a comment

Choose a reason for hiding this comment

DarkLight1337 commented Feb 22, 2025 • edited Loading

khluu commented Feb 20, 2025 •

edited by github-actions bot

Loading

DarkLight1337 commented Feb 22, 2025 •

edited

Loading