[Misc] improve example script output

[reidliu41]

improve the example scripts output, make it easier to view:

===> $ python examples/offline_inference/basic/basic.py
INFO 03-26 13:39:39 [llm_engine.py:447] init engine (profile, create kv cache, warmup model) took 0.14 seconds
Processed prompts:   0%|  | 0/4 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]WARNING 03-26 13:39:39 [cpu.py:154] Pin memory is not supported on CPU.
Processed prompts: 100%|█| 4/4 [00:00<00:00, 21.02it/s, est. speed input: 136.62 toks/s, output: 336
Prompt: 'Hello, my name is', Generated text: '\nRobert.\nI am the founder of the Peace Corps.\nI love'
Prompt: 'The president of the United States is', Generated text: ' discussing the idea of a foreign policy that he and his family could create in the'
Prompt: 'The capital of France is', Generated text: ' still under siege.\n\nFrance’s army is still on the rise'
Prompt: 'The future of AI is', Generated text: " in the internet, and I'm sure some of the best people in the world"

---> $ python examples/offline_inference/basic/basic.py
Generated Outputs:
------------------------------------------------------------
Prompt:    'Hello, my name is'
Output:    ' William, I live in California and I have a 3 year old girl I can'
------------------------------------------------------------
Prompt:    'The president of the United States is'
Output:    ' alleging that the American people are being exploited by the world’s best-'
------------------------------------------------------------
Prompt:    'The capital of France is'
Output:    ' losing its economic sprits right after the former French colony of Britain announced a'
------------------------------------------------------------
Prompt:    'The future of AI is'
Output:    ' with Deep Learning\n\nA year ago I was living in the throes of'
------------------------------------------------------------

===> $ python examples/offline_inference/basic/classify.py
WARNING 03-26 15:08:57 [cpu.py:154] Pin memory is not supported on CPU.
Processed prompts: 100%|█| 4/4 [00:00<00:00, 16.66it/s, est. speed input: 0.00 toks/s, output: 0.00
Prompt: 'Hello, my name is' | Class Probabilities: [nan, nan] (size=2)
Prompt: 'The president of the United States is' | Class Probabilities: [nan, nan] (size=2)
Prompt: 'The capital of France is' | Class Probabilities: [nan, nan] (size=2)
Prompt: 'The future of AI is' | Class Probabilities: [nan, nan] (size=2)

---> $ python examples/offline_inference/basic/classify.py
Processed prompts: 100%|█| 4/4 [00:00<00:00, 19.56it/s, est. speed input: 0.00 toks/s, output: 0.00

Generated Outputs:
------------------------------------------------------------
Prompt: 'Hello, my name is'
Class Probabilities: [nan, nan] (size=2)
------------------------------------------------------------
Prompt: 'The president of the United States is'
Class Probabilities: [nan, nan] (size=2)
------------------------------------------------------------
Prompt: 'The capital of France is'
Class Probabilities: [nan, nan] (size=2)
------------------------------------------------------------
Prompt: 'The future of AI is'
Class Probabilities: [nan, nan] (size=2)
------------------------------------------------------------

===> $ python examples/offline_inference/basic/embed.py
Processed prompts:   0%|  | 0/4 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]WARNING 03-26 15:18:33 [cpu.py:154] Pin memory is not supported on CPU.
Processed prompts: 100%|█| 4/4 [00:02<00:00,  1.97it/s, est. speed input: 0.00 toks/s, output: 0.00
Prompt: 'Hello, my name is' | Embeddings: [0.00983428955078125, 0.0245513916015625, -0.0177764892578125, -0.0018129348754882812, 0.01038360595703125, -0.0186309814453125, -0.01548004150390625, 0.016021728515625, 0.00910186767578125, 0.0014438629150390625, 0.020843505859375, -0.00797271728515625, 0.01554107666015625, 0.0028781890869140625, 0.004608154296875, 0.0038280487060546875, ...] (size=4096)
Prompt: 'The president of the United States is' | Embeddings: [-0.001514434814453125, -0.005222320556640625, -0.020233154296875, 0.0098114013671875, 0.0006422996520996094, -0.028472900390625, -0.001529693603515625, 0.01410675048828125, 0.0258331298828125, -0.0010156631469726562, 0.01971435546875, -0.0019130706787109375, 0.0229644775390625, 0.0010013580322265625, 0.01061248779296875, -0.010162353515625, ...] (size=4096)
Prompt: 'The capital of France is' | Embeddings: [0.01332855224609375, 0.007160186767578125, -0.0144805908203125, 0.0027446746826171875, 0.016937255859375, -0.01611328125, -0.00689697265625, 0.009979248046875, 0.037384033203125, 0.00313568115234375, 0.0032405853271484375, -0.00824737548828125, 0.00888824462890625, 0.0085296630859375, -0.0025272369384765625, 0.0007991790771484375, ...] (size=4096)
Prompt: 'The future of AI is' | Embeddings: [0.0023555755615234375, -0.01322174072265625, 0.004413604736328125, 0.00464630126953125, 0.005260467529296875, 0.0038928985595703125, -0.005519866943359375, 0.0101776123046875, 0.032318115234375, 0.0096435546875, -0.00039196014404296875, -0.007373809814453125, 0.0139312744140625, 0.005809783935546875, 0.00994873046875, -0.0034313201904296875, ...] (size=4096)

---> $ python examples/offline_inference/basic/embed.py
Generated Outputs:
------------------------------------------------------------
Prompt: 'Hello, my name is'
Embeddings: [0.00983428955078125, 0.0245513916015625, -0.0177764892578125, -0.0018129348754882812, 0.01038360595703125, -0.0186309814453125, -0.01548004150390625, 0.016021728515625, 0.00910186767578125, 0.0014438629150390625, 0.020843505859375, -0.00797271728515625, 0.01554107666015625, 0.0028781890869140625, 0.004608154296875, 0.0038280487060546875, ...] (size=4096)
------------------------------------------------------------
Prompt: 'The president of the United States is'
Embeddings: [-0.001514434814453125, -0.005222320556640625, -0.020233154296875, 0.0098114013671875, 0.0006422996520996094, -0.028472900390625, -0.001529693603515625, 0.01410675048828125, 0.0258331298828125, -0.0010156631469726562, 0.01971435546875, -0.0019130706787109375, 0.0229644775390625, 0.0010013580322265625, 0.01061248779296875, -0.010162353515625, ...] (size=4096)
------------------------------------------------------------
Prompt: 'The capital of France is'
Embeddings: [0.01332855224609375, 0.007160186767578125, -0.0144805908203125, 0.0027446746826171875, 0.016937255859375, -0.01611328125, -0.00689697265625, 0.009979248046875, 0.037384033203125, 0.00313568115234375, 0.0032405853271484375, -0.00824737548828125, 0.00888824462890625, 0.0085296630859375, -0.0025272369384765625, 0.0007991790771484375, ...] (size=4096)
------------------------------------------------------------
Prompt: 'The future of AI is'
Embeddings: [0.0023555755615234375, -0.01322174072265625, 0.004413604736328125, 0.00464630126953125, 0.005260467529296875, 0.0038928985595703125, -0.005519866943359375, 0.0101776123046875, 0.032318115234375, 0.0096435546875, -0.00039196014404296875, -0.007373809814453125, 0.0139312744140625, 0.005809783935546875, 0.00994873046875, -0.0034313201904296875, ...] (size=4096)
------------------------------------------------------------

===> python examples/offline_inference/basic/score.py
Processed prompts: 100%|█| 2/2 [00:00<00:00, 19.89it/s, est. speed input: 0.00 toks/s, output: 0.00
Pair: ['What is the capital of France?', 'The capital of Brazil is Brasilia.'] | Score: 0.001094818115234375
Pair: ['What is the capital of France?', 'The capital of France is Paris.'] | Score: 1.0

---> python examples/offline_inference/basic/score.py
WARNING 03-26 15:33:21 [cpu.py:154] Pin memory is not supported on CPU.
Processed prompts: 100%|█| 2/2 [00:00<00:00, 20.83it/s, est. speed input: 0.00 toks/s, output: 0.00

Generated Outputs:
------------------------------------------------------------
Pair: ['What is the capital of France?', 'The capital of Brazil is Brasilia.']
Score: 0.001094818115234375
------------------------------------------------------------
Pair: ['What is the capital of France?', 'The capital of France is Paris.']
Score: 1.0
------------------------------------------------------------

===> $ python examples/offline_inference/basic/chat.py
================================================================================
INFO 03-26 15:35:11 [chat_utils.py:379] Detected the chat template content format to be 'string'. You can set `--chat-template-content-format` to override this.
WARNING 03-26 15:35:12 [cpu.py:154] Pin memory is not supported on CPU.
Prompt: '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nCutting Knowledge Date: December 2023\nToday Date: 26 Mar 2025\n\nYou are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nHello<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\nHello! How can I assist you today?<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nWrite an essay about the importance of higher education.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n'
Generated text: 'The Importance of Higher Education\n\nHigher education is a cornerstone of modern society, playing'
--------------------------------------------------------------------------------
Processed prompts:   0%| | 0/10 [00:00<?, ?it/s, est. speed input: 0.00 toks/s, output: 0.00 toks/s]INFO 03-26 15:35:18 [metrics.py:481] Avg prompt throughput: 90.4 tokens/s, Avg generation throughput: 3.0 tokens/s, Running: 10 reqs, Swapped: 0 reqs, Pending: 0 reqs, GPU KV cache usage: 0.6%, CPU KV cache usage: 0.0%.
Processed prompts: 100%|█| 10/10 [00:06<00:00,  1.47it/s, est. speed input: 104.23 toks/s, output: 2
Prompt: '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nCutting Knowledge Date: December 2023\nToday Date: 26 Mar 2025\n\nYou are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nHello<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\nHello! How can I assist you today?<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nWrite an essay about the importance of higher education.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n'
Generated text: 'The Importance of Higher Education: Unlocking the Potential of the Next Generation\n\nHigher'
Prompt: '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nCutting Knowledge Date: December 2023\nToday Date: 26 Mar 2025\n\nYou are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nHello<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\nHello! How can I assist you today?<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nWrite an essay about the importance of higher education.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n'
Generated text: 'The Importance of Higher Education: Unlocking Human Potential\n\nHigher education is a vital'

---> $ python examples/offline_inference/basic/chat.py
Generated Outputs:
--------------------------------------------------------------------------------
Prompt: '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nCutting Knowledge Date: December 2023\nToday Date: 26 Mar 2025\n\nYou are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nHello<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\nHello! How can I assist you today?<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nWrite an essay about the importance of higher education.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n'

Generated text: 'The Importance of Higher Education: Unlocking Human Potential\n\nHigher education is a crucial'
--------------------------------------------------------------------------------
Prompt: '<|begin_of_text|><|start_header_id|>system<|end_header_id|>\n\nCutting Knowledge Date: December 2023\nToday Date: 26 Mar 2025\n\nYou are a helpful assistant<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nHello<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\nHello! How can I assist you today?<|eot_id|><|start_header_id|>user<|end_header_id|>\n\nWrite an essay about the importance of higher education.<|eot_id|><|start_header_id|>assistant<|end_header_id|>\n\n'

Generated text: 'The Importance of Higher Education: Unlocking the Potential of the Future\n\nHigher education'
--------------------------------------------------------------------------------
```

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[Isotr0py]

Thanks!