-
Notifications
You must be signed in to change notification settings - Fork 531
[v0.9.1][Bugfix] Fix guided decoding invalid backend #2645
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[v0.9.1][Bugfix] Fix guided decoding invalid backend #2645
Conversation
Signed-off-by: shen-shanshan <467638484@qq.com>
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Code Review
This pull request addresses a bug in the guided decoding tests where an invalid backend, guidance:disable-any-whitespace, was causing failures when running with VLLM_USE_V1=1. The fix correctly replaces the invalid backend with guidance, which resolves the NotImplementedError and allows the tests to pass as expected. The change is minimal, targeted, and effectively resolves the issue.
|
so just a test fixed? Why it was failure in history CI? |
I am also comfused about this. 😂 How does the CI passed in history v0.9.1 PR? |
| f"{guided_decoding_backend} will fall back to outlines, skip it") | ||
| if guided_decoding_backend == "outlines": | ||
| pytest.skip( | ||
| f"{guided_decoding_backend} will take up too much time for json " |
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
How much time does this case cost? if it is reasonable, I think we'd better keep it running, as the CI on v0.9.1-dev actually is not too much
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
maybe 10+ minutes...
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
Okay, then let's keep it skipped
What this PR does / why we need it?
Run:
Come across an error:
Does this PR introduce any user-facing change?
How was this patch tested?
Run:
Output: