[CI] Disable flaky structured output test from CI #24366
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
Contributor
There was a problem hiding this comment.
Code Review
This pull request disables a flaky test to unblock the CI pipeline. While this is a practical short-term solution, I've added a high-severity comment suggesting to create and link a tracking issue. This will ensure that the flaky test is not forgotten and is eventually fixed and re-enabled, which is crucial for maintaining long-term code quality and test coverage.
russellb
approved these changes
Sep 6, 2025
github-actions
bot
added
the
ready
Collaborator
|
Shall we create an issue to track the progress? |
Collaborator
|
Or we never want to bring it back :-) |
Member
Author
4 tasks
eicherseiji
pushed a commit
to eicherseiji/vllm
that referenced
this pull request
Sep 9, 2025
Signed-off-by: Roger Wang <hey@rogerw.io>
skyloevil
pushed a commit
to skyloevil/vllm
that referenced
this pull request
Sep 13, 2025
Signed-off-by: Roger Wang <hey@rogerw.io>
FeiDaLI
pushed a commit
to FeiDaLI/vllm
that referenced
this pull request
Sep 25, 2025
Signed-off-by: Roger Wang <hey@rogerw.io>
xuebwang-amd
pushed a commit
to xuebwang-amd/vllm
that referenced
this pull request
Oct 10, 2025
Signed-off-by: Roger Wang <hey@rogerw.io> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
xuebwang-amd
pushed a commit
to xuebwang-amd/vllm
that referenced
this pull request
Oct 24, 2025
Signed-off-by: Roger Wang <hey@rogerw.io> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
3 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Purpose
v1/entrypoints/llm/test_struct_output_generate.py::test_structured_output[mistralai/Ministral-8B-Instruct-2410-lm-format-enforcer-auto-None]This particular test has been flaky on main CI for more than 2 weeks and has blocked auto-merge for quite a number of pull requests but it doesn't have an owner to fix. This PR disables it from main CI since this is a test on a particular model with a particular feature so it should be low risk.
Test Plan
Test Result
Essential Elements of an Effective PR Description Checklist
supported_models.mdandexamplesfor a new model.