-
-
Notifications
You must be signed in to change notification settings - Fork 11k
[BugFix] Fix --disable-log-stats in V1 server mode
#17600
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[BugFix] Fix --disable-log-stats in V1 server mode
#17600
Conversation
|
👋 Hi! Thank you for contributing to the vLLM project. 💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels. Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging. To run CI, PR reviewers can either: Add 🚀 |
Also sum gpu blocks across DP ranks when reporting the num_gpu_blocks metric. Signed-off-by: Nick Hill <nhill@redhat.com>
371d45c to
5913eb8
Compare
Signed-off-by: Nick Hill <nhill@redhat.com>
|
I think the test failure is also happening on main. I will look into that too |
Signed-off-by: Nick Hill <nhill@redhat.com>
…#1249) [BugFix] Fix --disable-log-stats in V1 server mode vllm-project#17600
Also sum gpu blocks across DP ranks when reporting the
num_gpu_blocksmetric.Both introduced by #15755
There is a test for this but it's not running on V1 yet. Will look at changing that: https://github.com/vllm-project/vllm/blob/main/tests/metrics/test_metrics.py