[Core] Streamline some structured output related code #26737
Merged
Conversation
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
njhill
requested review from
ApostaC,
NickLucche,
WoosukKwon,
aarnphm,
alexm-redhat,
benchislett,
comaniac,
heheda12345,
mgoin,
robertgshaw2-redhat,
russellb and
ywang96
as code owners
![gemini-code-assist[bot]](https://avatars.githubusercontent.com/in/956858?s=60&v=4){kind=small}
There was a problem hiding this comment.
Code Review
This pull request introduces several beneficial refactorings to streamline the structured output related code. Key improvements include simplifying the structured_output_request_ids data structure from a dictionary to a list, making Request.use_structured_output a derived property for better state management, and improving the encapsulation of StructuredOutputRequest. These changes enhance code clarity and maintainability. I've identified one issue in gpu_model_runner.py where a conditional check is always true, which should be corrected.
![chatgpt-codex-connector[bot]](https://avatars.githubusercontent.com/in/1144995?s=60&v=4){kind=small}
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Signed-off-by: Nick Hill <nhill@redhat.com>
njhill
force-pushed
the
streamline-struct-output
branch
from
fbab8e3 to
ec7ceda
Compare
Signed-off-by: Nick Hill <nhill@redhat.com>
Signed-off-by: Nick Hill <nhill@redhat.com>
russellb
approved these changes
Oct 14, 2025
| # Dict of request ids to their index within the batch | ||
| # for filling the next token bitmask | ||
| structured_output_request_ids: dict[str, int] | ||
| # ids of structured outputs requests included in the bitmask, in order. |
There was a problem hiding this comment.
Suggested change
| # ids of structured outputs requests included in the bitmask, in order. | |
| # ids of structured outputs requests included in the bitmask. | |
| # The index of the id in this list corresponds to the index into | |
| # grammar_bitmask. |
is this right?
There was a problem hiding this comment.
Actually not necessarily in spec decoding case. The requests will be in the order of the list but there may be more than one row per request so the index in the list won't match index into the bitmask. It would be true in non-spec decode case though.
There was a problem hiding this comment.
Oh, right -- clarifying the order definition here would be nice in any case
There was a problem hiding this comment.
I will do this in follow-on PR to avoid running the whole CI again!
Jonahcb
pushed a commit
to Jonahcb/vllm
that referenced
this pull request
Oct 15, 2025
…6737) Signed-off-by: Nick Hill <nhill@redhat.com> Signed-off-by: Jonah Bernard <jb2528@cornell.edu>
bbartels
pushed a commit
to bbartels/vllm
that referenced
this pull request
Oct 16, 2025
…6737) Signed-off-by: Nick Hill <nhill@redhat.com> Signed-off-by: bbartels <benjamin@bartels.dev>
njhill
added a commit
to njhill/vllm
that referenced
this pull request
Oct 17, 2025
This is a follow-on to PRs vllm-project#26737 and vllm-project#26961 to add some clarifying comments that were suggested in review. Signed-off-by: Nick Hill <nhill@redhat.com>
lywa1998
pushed a commit
to lywa1998/vllm
that referenced
this pull request
Oct 20, 2025
…6737) Signed-off-by: Nick Hill <nhill@redhat.com>
alhridoy
pushed a commit
to alhridoy/vllm
that referenced
this pull request
Oct 24, 2025
…6737) Signed-off-by: Nick Hill <nhill@redhat.com>
wangxiyuan
pushed a commit
to vllm-project/vllm-ascend
that referenced
this pull request
Oct 24, 2025
### What this PR does / why we need it? This is the step 1 of refactoring code to adapt with vllm main, and this pr aligned with vllm-project/vllm@17c540a 1. refactor deepseek to the latest code arch as of vllm-project/vllm@17c540a 2. bunches of fixes due to vllm changes - Fix `AscendScheduler` `__post_init__`, caused by vllm-project/vllm#25075 - Fix `AscendScheduler` init got an unexpected arg `block_size`, caused by vllm-project/vllm#26296 - Fix `KVCacheManager` `get_num_common_prefix_blocks` arg, caused by vllm-project/vllm#23485 - Fix `MLAAttention` import,caused by vllm-project/vllm#25103 - Fix `SharedFusedMoE` import, caused by vllm-project/vllm#26145 - Fix `LazyLoader` improt, caused by vllm-project/vllm#27022 - Fix `vllm.utils.swap_dict_values` improt, caused by vllm-project/vllm#26990 - Fix `Backend` enum import, caused by vllm-project/vllm#25893 - Fix `CompilationLevel` renaming to `CompilationMode` issue introduced by vllm-project/vllm#26355 - Fix fused_moe ops, caused by vllm-project/vllm#24097 - Fix bert model because of `inputs_embeds`, caused by vllm-project/vllm#25922 - Fix MRope because of `get_input_positions_tensor` to `get_mrope_input_positions`, caused by vllm-project/vllm#24172 - Fix `splitting_ops` changes introduced by vllm-project/vllm#25845 - Fix multi-modality changes introduced by vllm-project/vllm#16229 - Fix lora bias dropping issue introduced by vllm-project/vllm#25807 - Fix structured ouput break introduced by vllm-project/vllm#26737 ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? CI passed with existing test. - vLLM version: v0.11.0rc3 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0 --------- Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: Icey <1790571317@qq.com> Co-authored-by: Icey <1790571317@qq.com>
xuebwang-amd
pushed a commit
to xuebwang-amd/vllm
that referenced
this pull request
Oct 24, 2025
…6737) Signed-off-by: Nick Hill <nhill@redhat.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
xuebwang-amd
pushed a commit
to xuebwang-amd/vllm
that referenced
this pull request
Oct 24, 2025
…6737) Signed-off-by: Nick Hill <nhill@redhat.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>
0xrushi
pushed a commit
to 0xrushi/vllm
that referenced
this pull request
Oct 26, 2025
…6737) Signed-off-by: Nick Hill <nhill@redhat.com> Signed-off-by: 0xrushi <6279035+0xrushi@users.noreply.github.com>
0xrushi
pushed a commit
to 0xrushi/vllm
that referenced
this pull request
Oct 26, 2025
…6737) Signed-off-by: Nick Hill <nhill@redhat.com> Signed-off-by: 0xrushi <6279035+0xrushi@users.noreply.github.com>
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Simplify some data structures, redundant fields, etc.
Things noticed while reading through the code to understand it.