[Misc] Replace os environ to monkeypatch in test suite #14516

t-sibiraj · 2025-03-09T09:58:34Z

Convert all os.environ(xxx) to monkeypatch.setenv in test suite

Submitting pull request again because I forget to sign my original pull request

Co-authored-by: Aaron Pham contact@aarnphm.xyz
Signed-off-by: Aaron Pham contact@aarnphm.xyz

github-actions · 2025-03-09T09:58:45Z

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

tests/basic_correctness/test_cumem.py

tests/test_utils.py

DarkLight1337 · 2025-03-09T15:12:01Z

Please fix the pre-commit errors

robertgshaw2-redhat · 2025-03-09T16:58:14Z

tests/models/decoder_only/language/test_gptq_marlin.py

you cant pass monkeypatch to a module level fixture

robertgshaw2-redhat · 2025-03-09T16:58:21Z

tests/models/decoder_only/vision_language/test_models.py

robertgshaw2-redhat · 2025-03-09T16:58:27Z

tests/models/decoder_only/vision_language/test_phi3v.py

robertgshaw2-redhat · 2025-03-09T20:06:04Z

I would suggest installing the pre-commit which will automatically sign off for you. Run ./format.sh locally and it will give you instructions

t-sibiraj · 2025-03-11T17:55:50Z

I would suggest installing the pre-commit which will automatically sign off for you. Run ./format.sh locally and it will give you instructions

okie will test locally first

tests/models/decoder_only/vision_language/test_models.py

Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>

Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>

… variable removal Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>

Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>

Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>

…ld push the changes and pass the pre-commit check Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>

Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>

t-sibiraj · 2025-03-15T14:47:50Z

@t-sibiraj you can apply this patch befor 4932bcd

14516.patch

Thank you so much I will apply the patch

aarnphm · 2025-03-15T16:41:48Z

hmm, can you add me as collaborator to your branch? I can help with expedite this PR and fix CI error :)

t-sibiraj · 2025-03-15T17:07:09Z

hmm, can you add me as collaborator to your branch? I can help with expedite this PR and fix CI error :)

Thank you I really appreciate your help. I have sent you an invite.

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

aarnphm · 2025-03-16T16:20:24Z

@t-sibiraj can you add the following to the PR description?

Co-authored-by: Aaron Pham <contact@aarnphm.xyz>
Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

Thanks

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

DarkLight1337 · 2025-03-17T03:34:14Z

Done, and thanks for fixing this!

DarkLight1337 · 2025-03-17T03:35:48Z

The CI failures are unrelated so let's merge this

…14516) Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com> Signed-off-by: Aaron Pham <contact@aarnphm.xyz> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Co-authored-by: Aaron Pham <contact@aarnphm.xyz> Signed-off-by: DefTruth <31974251+DefTruth@users.noreply.github.com>

ProExpertProg · 2025-03-27T21:44:13Z

This PR broke the PyTorch Fullgraph tests:

[2025-03-16T17:00:51Z] Running 2 items in this shard: tests/compile/test_full_graph.py::test_full_graph[model_info0-2], tests/compile/test_full_graph.py::test_full_graph[model_info0-3]
--
  | [2025-03-16T17:00:51Z]
  | [2025-03-16T17:00:51Z] compile/test_full_graph.py::test_full_graph[model_info0-2] SKIPPED (...)
  | [2025-03-16T17:00:51Z] compile/test_full_graph.py::test_full_graph[model_info0-3] SKIPPED (...)

The fixture cannot be used in this simple way to generate an array of parameters

…14516) Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com> Signed-off-by: Aaron Pham <contact@aarnphm.xyz> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Co-authored-by: Aaron Pham <contact@aarnphm.xyz> Signed-off-by: Louis Ulmer <ulmerlouis@gmail.com>

Pass pooling_metadata to pooler head in gritlm. This was broken by PR vllm-project#16331 broke gritlm. PR vllm-project#14516 broke gritlm tests due to changing xformers to flash_atnn Signed-off-by: Pooya Davoodi <pooya.davoodi@parasail.io>

…14516) Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com> Signed-off-by: Aaron Pham <contact@aarnphm.xyz> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Co-authored-by: Aaron Pham <contact@aarnphm.xyz>

…14516) Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com> Signed-off-by: Aaron Pham <contact@aarnphm.xyz> Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Co-authored-by: Aaron Pham <contact@aarnphm.xyz> Signed-off-by: Mu Huai <tianbowen.tbw@antgroup.com>

t-sibiraj requested review from DarkLight1337, WoosukKwon, mgoin, russellb, tlrmchlsmth and ywang96 as code owners March 9, 2025 09:58

mergify bot added the v1 label Mar 9, 2025

DarkLight1337 reviewed Mar 9, 2025

View reviewed changes

tests/basic_correctness/test_cumem.py Outdated Show resolved Hide resolved

DarkLight1337 reviewed Mar 9, 2025

View reviewed changes

tests/test_utils.py Outdated Show resolved Hide resolved

DarkLight1337 approved these changes Mar 9, 2025

View reviewed changes

robertgshaw2-redhat reviewed Mar 9, 2025

View reviewed changes

tests/models/decoder_only/language/test_gptq_marlin.py Outdated

Copy link

Collaborator

robertgshaw2-redhat Mar 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

you cant pass monkeypatch to a module level fixture

robertgshaw2-redhat reviewed Mar 9, 2025

View reviewed changes

tests/models/decoder_only/vision_language/test_models.py Outdated

Copy link

Collaborator

robertgshaw2-redhat Mar 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

robertgshaw2-redhat reviewed Mar 9, 2025

View reviewed changes

tests/models/decoder_only/vision_language/test_phi3v.py Outdated

Copy link

Collaborator

robertgshaw2-redhat Mar 9, 2025

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

ditto

t-sibiraj force-pushed the replace-os-environ-monkeypatch branch from a6ade47 to 1e64dc0 Compare March 12, 2025 07:14

DarkLight1337 reviewed Mar 12, 2025

View reviewed changes

tests/models/decoder_only/vision_language/test_models.py Outdated Show resolved Hide resolved

DarkLight1337 enabled auto-merge (squash) March 12, 2025 08:10

github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Mar 12, 2025

t-sibiraj and others added 6 commits March 12, 2025 16:25

Replace os.environ with monkeypatch in test suite

4e69ecc

Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>

Replace os.environ with monkeypatch in test suite

0f9ba59

Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>

Update tests/basic_correctness/test_cumem.py

44c73bb

Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>

fix: Use raising=True in monkeypatch.delenv for safer environment…

5f28f2d

… variable removal Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>

doing a signed commit

8436a5f

Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>

remove unnecessary spacing

b791b51

Co-authored-by: Cyrus Leung <cyrus.tl.leung@gmail.com> Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>

auto-merge was automatically disabled March 12, 2025 08:26
Head branch was pushed to by a user without write access

t-sibiraj force-pushed the replace-os-environ-monkeypatch branch from e9c8188 to b791b51 Compare March 12, 2025 08:26

Intentionally added and removed a space to test this commit, so I cou…

6971e83

…ld push the changes and pass the pre-commit check Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>

Apply patch before 4932bcd

7139925

Signed-off-by: sibi <85477603+t-sibiraj@users.noreply.github.com>

t-sibiraj force-pushed the replace-os-environ-monkeypatch branch from 4932bcd to 7139925 Compare March 15, 2025 14:47

aarnphm added 6 commits March 15, 2025 13:24

fix: correct tests

48b9bf4

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

fix: correct types

bb8540c

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

fix: ignore otel imports mypy

7430fc6

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

fix: types

9191625

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

fix: distributed tests with dispatch

e324fb2

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

chore: fix tests

5266dff

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

mergify bot added the ci/build label Mar 16, 2025

fix: precommit error

a46f271

Signed-off-by: Aaron Pham <contact@aarnphm.xyz>

DarkLight1337 enabled auto-merge (squash) March 17, 2025 03:34

vllm-bot merged commit a73e183 into vllm-project:main Mar 17, 2025
55 of 58 checks passed

aarnphm deleted the replace-os-environ-monkeypatch branch March 17, 2025 04:53

ProExpertProg mentioned this pull request Mar 27, 2025

[Bugfix] [torch.compile] Add Dynamo metrics context during compilation #15639

Merged

pooyadavoodi mentioned this pull request Apr 15, 2025

[Bugfix] Fix broken GritLM model and tests (missing pooling_metadata) #16631

Merged

ckhordiasma mentioned this pull request Apr 17, 2025

[do not merge] pr test for nm changes into 2.20 red-hat-data-services/vllm#107

Closed

Uh oh!

[Misc] Replace os environ to monkeypatch in test suite #14516

[Misc] Replace os environ to monkeypatch in test suite #14516

Uh oh!

Conversation

t-sibiraj commented Mar 9, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

github-actions bot commented Mar 9, 2025

Uh oh!

Uh oh!

Uh oh!

DarkLight1337 commented Mar 9, 2025

Uh oh!

robertgshaw2-redhat Mar 9, 2025

Choose a reason for hiding this comment

Uh oh!

robertgshaw2-redhat Mar 9, 2025

Choose a reason for hiding this comment

Uh oh!

robertgshaw2-redhat Mar 9, 2025

Choose a reason for hiding this comment

Uh oh!

robertgshaw2-redhat commented Mar 9, 2025

Uh oh!

t-sibiraj commented Mar 11, 2025

Uh oh!

Uh oh!

t-sibiraj commented Mar 15, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

aarnphm commented Mar 15, 2025

Uh oh!

t-sibiraj commented Mar 15, 2025

Uh oh!

aarnphm commented Mar 16, 2025

Uh oh!

DarkLight1337 commented Mar 17, 2025

Uh oh!

DarkLight1337 commented Mar 17, 2025

Uh oh!

Uh oh!

ProExpertProg commented Mar 27, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

t-sibiraj commented Mar 9, 2025 •

edited by github-actions bot

Loading

t-sibiraj commented Mar 15, 2025 •

edited

Loading