Skip to content

[V0 deprecation] Guided decoding #21347

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 12 commits into from
Jul 29, 2025
Merged

[V0 deprecation] Guided decoding #21347

merged 12 commits into from
Jul 29, 2025

Conversation

@rzabarazesh
Copy link
Collaborator

@rzabarazesh rzabarazesh commented Jul 22, 2025 •
edited by github-actions bot
Loading

Essential Elements of an Effective PR Description Checklist

  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

Remove v0 guided decoding code.

Test Plan

Test Result

(Optional) Documentation Update

gemini-code-assist[bot]
gemini-code-assist bot reviewed Jul 22, 2025
View reviewed changes
Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request effectively removes the v0 guided decoding functionality, which simplifies the codebase significantly. The changes mostly consist of deleting old files and removing references to the deprecated v0 guided decoding API, which has been done correctly across the board.

However, I've identified a critical inconsistency regarding the outlines guided decoding backend. While its implementation has been removed, it's still listed as a valid backend in the configuration, leading to contradictory behavior in the tests. My review includes a detailed comment and a suggestion to resolve this ambiguity.

@rzabarazesh rzabarazesh force-pushed the guided_decoding_deprecation_v0 branch from 8be0e76 to 6019893 Compare July 22, 2025 04:08
@github-actions
Copy link

github-actions bot commented Jul 22, 2025

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

@rzabarazesh rzabarazesh force-pushed the guided_decoding_deprecation_v0 branch from 6019893 to 6eaa86b Compare July 22, 2025 06:20
@mergify mergify bot added the ci/build label Jul 22, 2025
@rzabarazesh rzabarazesh force-pushed the guided_decoding_deprecation_v0 branch from 6eaa86b to b594e93 Compare July 22, 2025 06:22
@rzabarazesh rzabarazesh changed the title Guided decoding v0 deprecation [Core] Guided decoding v0 deprecation Jul 22, 2025
@rzabarazesh rzabarazesh force-pushed the guided_decoding_deprecation_v0 branch from b594e93 to 0d62990 Compare July 22, 2025 06:39
@rzabarazesh rzabarazesh marked this pull request as ready for review July 22, 2025 14:09
hmellor and others added 6 commits July 28, 2025 20:49
@hmellor @rzabarazesh
Reinstate and update lazy outlines test
e0658b0 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@hmellor @rzabarazesh
Reinstate and update guided generate test (not currently passing)
ac2543d 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@hmellor @rzabarazesh
No tokenizer tests are changed
19703c7 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@hmellor @rzabarazesh
Re enable tests that should be run on V1
fe7b35e 
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@rzabarazesh
Remove test_guided_generate.py
0664e5b 
Signed-off-by: Reza Barazesh <rezabarazesh@meta.com>
@rzabarazesh
Remove lm-format-enforcer reference from test_oracle.py
c5e1d9b 
Signed-off-by: Reza Barazesh <rezabarazesh@meta.com>
@rzabarazesh rzabarazesh force-pushed the guided_decoding_deprecation_v0 branch from ebb4ca5 to c5e1d9b Compare July 29, 2025 03:49
@hmellor
Copy link
Member

hmellor commented Jul 29, 2025

@russellb thanks for confirming that tests/entrypoints/llm/test_guided_generate.py does not need to be ported to V1

@hmellor hmellor dismissed their stale review July 29, 2025 07:46

Originally wasn't sure about some test deletion, it's been confirmed that we can delete them

@hmellor hmellor moved this to In Progress in V0 Deprecation Jul 29, 2025
@hmellor hmellor mentioned this pull request Jul 29, 2025
Closed
@vllm-bot vllm-bot merged commit 37efc63 into vllm-project:main Jul 29, 2025
98 of 100 checks passed
@github-project-automation github-project-automation bot moved this from In Progress to Done in V0 Deprecation Jul 29, 2025
liuyumoye pushed a commit to liuyumoye/vllm that referenced this pull request Jul 31, 2025
@rzabarazesh @hmellor
[V0 deprecation] Guided decoding (vllm-project#21347)
2b37df3 
Signed-off-by: Reza Barazesh <rezabarazesh@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
x22x22 pushed a commit to x22x22/vllm that referenced this pull request Aug 5, 2025
@rzabarazesh @hmellor @x22x22
[V0 deprecation] Guided decoding (vllm-project#21347)
9c7e98e 
Signed-off-by: Reza Barazesh <rezabarazesh@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: x22x22 <wadeking@qq.com>
Pradyun92 pushed a commit to Pradyun92/vllm that referenced this pull request Aug 6, 2025
@rzabarazesh @hmellor
[V0 deprecation] Guided decoding (vllm-project#21347)
84cadca 
Signed-off-by: Reza Barazesh <rezabarazesh@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
npanpaliya pushed a commit to odh-on-pz/vllm-upstream that referenced this pull request Aug 6, 2025
@rzabarazesh @hmellor @npanpaliya
[V0 deprecation] Guided decoding (vllm-project#21347)
93ff769 
Signed-off-by: Reza Barazesh <rezabarazesh@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
jinzhen-lin pushed a commit to jinzhen-lin/vllm that referenced this pull request Aug 9, 2025
@rzabarazesh @hmellor @jinzhen-lin
[V0 deprecation] Guided decoding (vllm-project#21347)
b4a3f4e 
Signed-off-by: Reza Barazesh <rezabarazesh@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
noamgat pushed a commit to noamgat/vllm that referenced this pull request Aug 9, 2025
@rzabarazesh @hmellor @noamgat
[V0 deprecation] Guided decoding (vllm-project#21347)
71c6ba1 
Signed-off-by: Reza Barazesh <rezabarazesh@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
paulpak58 pushed a commit to paulpak58/vllm that referenced this pull request Aug 13, 2025
@rzabarazesh @hmellor @paulpak58
[V0 deprecation] Guided decoding (vllm-project#21347)
3d52fd3 
Signed-off-by: Reza Barazesh <rezabarazesh@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Paul Pak <paulpak58@gmail.com>
diegocastanibm pushed a commit to diegocastanibm/vllm that referenced this pull request Aug 15, 2025
@rzabarazesh @hmellor @diegocastanibm
[V0 deprecation] Guided decoding (vllm-project#21347)
581b888 
Signed-off-by: Reza Barazesh <rezabarazesh@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Signed-off-by: Diego-Castan <diego.castan@ibm.com>
epwalsh pushed a commit to epwalsh/vllm that referenced this pull request Aug 28, 2025
@rzabarazesh @hmellor @epwalsh
[V0 deprecation] Guided decoding (vllm-project#21347)
b2f53ef 
Signed-off-by: Reza Barazesh <rezabarazesh@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
zhewenl pushed a commit to zhewenl/vllm that referenced this pull request Aug 28, 2025
@rzabarazesh @hmellor @zhewenl
[V0 deprecation] Guided decoding (vllm-project#21347)
bd1befc 
Signed-off-by: Reza Barazesh <rezabarazesh@meta.com>
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
Co-authored-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
@JoshBClemons JoshBClemons mentioned this pull request Sep 4, 2025
Merged
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@WoosukKwon WoosukKwon WoosukKwon approved these changes

@hmellor hmellor hmellor left review comments

@DarkLight1337 DarkLight1337 Awaiting requested review from DarkLight1337 DarkLight1337 is a code owner

@ywang96 ywang96 Awaiting requested review from ywang96 ywang96 is a code owner

@mgoin mgoin Awaiting requested review from mgoin

@russellb russellb Awaiting requested review from russellb

@aarnphm aarnphm Awaiting requested review from aarnphm aarnphm is a code owner

@robertgshaw2-redhat robertgshaw2-redhat Awaiting requested review from robertgshaw2-redhat robertgshaw2-redhat is a code owner

@simon-mo simon-mo Awaiting requested review from simon-mo simon-mo is a code owner

@youkaichao youkaichao Awaiting requested review from youkaichao

@tlrmchlsmth tlrmchlsmth Awaiting requested review from tlrmchlsmth

@houseroad houseroad Awaiting requested review from houseroad

@zhuohan123 zhuohan123 Awaiting requested review from zhuohan123

@alexm-redhat alexm-redhat Awaiting requested review from alexm-redhat

@comaniac comaniac Awaiting requested review from comaniac

@njhill njhill Awaiting requested review from njhill

+1 more reviewer

@gemini-code-assist gemini-code-assist[bot] gemini-code-assist[bot] left review comments

Reviewers whose approvals may not affect merge requirements

Assignees

No one assigned

Labels

ci/build frontend ready ONLY add when PR is ready to merge/full CI is needed structured-output v1

Projects

Structured Output
Status: Done
V0 Deprecation
Status: Done

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@rzabarazesh @WoosukKwon @hmellor @russellb @vllm-bot