Skip to content

[Misc] Update guided decoding logs to debug #15310

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom
Merged
merged 1 commit into from
Mar 24, 2025
Merged

[Misc] Update guided decoding logs to debug #15310

merged 1 commit into from
Mar 24, 2025

Conversation

@sfbemerk
Copy link
Contributor

@sfbemerk sfbemerk commented Mar 21, 2025 •
edited by github-actions bot
Loading

The json schema as part of guided decoding details can contain sensitive user inputs. I believe user inputs should not be logged when the engine is started with disable_log_requests=True.

These logs have existed for a while at DEBUG level but only made visible at INFO level with v0.8.0 (#12955). I would be fine with reverting the relevant line's log level back to DEBUG instead of my proposed change.

@github-actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

@sfbemerk sfbemerk changed the title [Bugfix] Hide guided decoding details if disable_log_requests is set. [Bugfix] Hide guided decoding detail logs if disable_log_requests is set. Mar 22, 2025
@simon-mo simon-mo requested review from aarnphm and russellb March 23, 2025 22:45
@robertgshaw2-redhat
Copy link
Collaborator

Should we just revert this to debug? Would rather not make these functions have more options if we can avoid it

@aarnphm
Copy link
Collaborator

aarnphm commented Mar 24, 2025

+1 with rob, let's just make it logger.debug

Log guided decoding details on DEBUG, not INFO.
10b2bf1 
Guided decoding json schema contains potentially sensitive user inputs. Do not log user inputs on INFO while ignoring `disable_log_requests=True`.

Signed-off-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
@sfbemerk
Copy link
Contributor Author

Okay, then let's keep things simple and just revert the logs to debug.
I updated the PR. Thanks for the quick review!

aarnphm
aarnphm approved these changes Mar 24, 2025
View reviewed changes
Copy link
Collaborator

@aarnphm aarnphm left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Can you also update the title to "[Misc] Update guided decoding logs to debug"

Thanks

@hmellor hmellor changed the title [Bugfix] Hide guided decoding detail logs if disable_log_requests is set. [Misc] Update guided decoding logs to debug Mar 24, 2025
hmellor
hmellor approved these changes Mar 24, 2025
View reviewed changes
Copy link
Member

@hmellor hmellor left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@hmellor hmellor enabled auto-merge (squash) March 24, 2025 10:36
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Mar 24, 2025
@sfbemerk
Copy link
Contributor Author

Thank you all!
I think the CI is failing for unrelated reasons. @hmellor could you please force-merge?

@vllm-bot vllm-bot merged commit cc8accf into vllm-project:main Mar 24, 2025
37 of 45 checks passed
erictang000 pushed a commit to erictang000/vllm that referenced this pull request Mar 25, 2025
@sfbemerk @erictang000
[Misc] Update guided decoding logs to debug (vllm-project#15310)
4ea8fb9 
Signed-off-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
Co-authored-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
wrmedford pushed a commit to wrmedford/vllm that referenced this pull request Mar 26, 2025
@sfbemerk @wrmedford
[Misc] Update guided decoding logs to debug (vllm-project#15310)
0ea7347 
Signed-off-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
Co-authored-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
Signed-off-by: Wes Medford <wryanmedford@gmail.com>
lulmer pushed a commit to lulmer/vllm that referenced this pull request Apr 7, 2025
@sfbemerk @lulmer
[Misc] Update guided decoding logs to debug (vllm-project#15310)
5478b5d 
Signed-off-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
Co-authored-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
Signed-off-by: Louis Ulmer <ulmerlouis@gmail.com>
@ckhordiasma ckhordiasma mentioned this pull request Apr 17, 2025
Closed
lk-chen pushed a commit to lk-chen/vllm that referenced this pull request Apr 29, 2025
@sfbemerk @lk-chen
[Misc] Update guided decoding logs to debug (vllm-project#15310)
bc02179 
Signed-off-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
Co-authored-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
shreyankg pushed a commit to shreyankg/vllm that referenced this pull request May 3, 2025
@sfbemerk @shreyankg
[Misc] Update guided decoding logs to debug (vllm-project#15310)
a164bd5 
Signed-off-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
Co-authored-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
RichardoMrMu pushed a commit to RichardoMrMu/vllm that referenced this pull request May 12, 2025
@sfbemerk
[Misc] Update guided decoding logs to debug (vllm-project#15310)
311065d 
Signed-off-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
Co-authored-by: Benjamin Merkel <benjamin.merkel@tngtech.com>
Signed-off-by: Mu Huai <tianbowen.tbw@antgroup.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

@hmellor hmellor hmellor approved these changes

@aarnphm aarnphm aarnphm approved these changes

@russellb russellb Awaiting requested review from russellb

Assignees

No one assigned

Labels

ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

5 participants

@sfbemerk @robertgshaw2-redhat @aarnphm @hmellor @vllm-bot