[Misc] Change buckets of histogram_iteration_tokens to [1, 8, 16, 32, 64, 128, 256, 512, 1024, 2048, 4096, 8096] to represent number of tokens

[sfc-gh-zhwang]

No description provided.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[youkaichao]

LGTM, thanks for the clean-up!

[youkaichao]

initially I added it in #11031 to track batchsize distribution, but later we mainly use VLLM_LOG_BATCHSIZE_INTERVAL environment variable for this purpose.

[DarkLight1337]

Please fix pre-commit

[DarkLight1337]

Pre-commit is still failing, can you fix it?

[sfc-gh-zhwang]

[2025-04-24T05:49:22Z] FAILED spec_decode/e2e/test_multistep_correctness.py::test_spec_decode_e2e_greedy_correctness_with_preemption[1-4-256-test_llm_kwargs1-baseline_llm_kwargs0-per_test_common_llm_kwargs0-common_llm_kwargs0] - AssertionError: function <function test_spec_decode_e2e_greedy_correctness_with_preemption at 0x7f4d58803880> failed when called with args () and kwargs {'vllm_runner': <class 'tests.conftest.VllmRunner'>, 'common_llm_kwargs': {'block_size': 8, 'num_gpu_blocks_override': 34, 'max_model_len': 272, 'enforce_eager': True}, 'per_test_common_llm_kwargs': {'model_name': 'JackFram/llama-160m'}, 'baseline_llm_kwargs': {}, 'test_llm_kwargs': {'speculative_config': {'model': 'JackFram/llama-68m', 'num_speculative_tokens': 5}, 'enable_chunked_prefill': True, 'max_num_batched_tokens': 4, 'max_num_seqs': 4}, 'batch_size': 4, 'output_len': 256, 'seed': 1}
--
  | [2025-04-24T05:49:22Z] FAILED spec_decode/e2e/test_multistep_correctness.py::test_spec_decode_different_block_size[1-32-2-test_llm_kwargs1-baseline_llm_kwargs0-per_test_common_llm_kwargs0-common_llm_kwargs0] - AssertionError: function <function test_spec_decode_different_block_size at 0x7f4d588039c0> failed when called with args () and kwargs {'vllm_runner': <class 'tests.conftest.VllmRunner'>, 'common_llm_kwargs': {'model_name': 'JackFram/llama-160m', 'enforce_eager': True}, 'per_test_common_llm_kwargs': {'block_size': 8}, 'baseline_llm_kwargs': {}, 'test_llm_kwargs': {'speculative_config': {'model': 'JackFram/llama-68m', 'num_speculative_tokens': 5}, 'enable_chunked_prefill': True, 'max_num_batched_tokens': 4, 'max_num_seqs': 4}, 'batch_size': 2, 'output_len': 32, 'seed': 1}

@DarkLight1337 i am curious why would this fail

[DarkLight1337]

The spec decode failure is unrelated to the PR. I think you just need to fix pre-commit. Make sure you merge from main as well

[DarkLight1337]

I have fixed pre-commit for you

[sfc-gh-zhwang]

@DarkLight1337 sorry didn't get time, thank you so much