Skip to content

Conversation

@YonatanGideoni
Copy link
Contributor

Fixes issue #4159 when vLLM is set to colocate mode. For server mode the user would have to pass --logprobs-mode processed_logprobs - should we add a message about this in the docs? A warning? What's best?

Bumps vLLM version to 0.10.2

Adds a test checking that the TIS ratio is reasonable for a lower temperature. Skipped by default as it requires vLLM

@qgallouedec
Copy link
Member

Thanks for the fix—super helpful!
I removed the test since we generally prefer testing the public API and keeping tests independent of internal logic. I’m not sure how to meaningfully verify whether the logprobs are correct, so by default I’d rather leave that untested.

@qgallouedec qgallouedec changed the title Have vLLM return processed (temperature scaled) log probs 🌡️ Have vLLM return processed (temperature scaled) log probs Oct 1, 2025
@HuggingFaceDocBuilderDev

The docs for this PR live here. All of your documentation changes will be reflected on that endpoint. The docs are available until 30 days after the last update.

@qgallouedec qgallouedec merged commit e086f07 into huggingface:main Oct 1, 2025
10 checks passed
qgallouedec added a commit that referenced this pull request Oct 6, 2025
Co-authored-by: Quentin Gallouédec <gallouedec.quentin@gmail.com>
Co-authored-by: Quentin Gallouédec <45557362+qgallouedec@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants