[Frontend][3/N] Improve all pooling task | Support binary embedding response

[noooop]

TL;DR

• support endianness: ["native", "big", "little"], native by default
• support bytes encoding_format, a very simple (but highly efficient) binary embedding response method
  • First, write metadata into the headers, then write all the binary data in order into Response body.
  • When reading, first read the headers, then read the body according to the offset into the corresponding tenser
• This API provides three benefits @uasan

1. Significant reduction in response size
2. No need for JSON.parse and Base64.decode
3. Possibility of stream processing of the response
4. Endianness customizations

Improve all pooling task

• [Model][0/N] Improve all pooling task | clean up #25817
• [Frontend][1/N] Improve all pooling task | Support FP16 Embedding Base64 (Still uses fp32 by default). #26414
• [Model][2/N] Improve all pooling task | Support multi-vector retrieval #25370
• [Frontend][3/N] Improve all pooling task | Support binary embedding response #27066
• [Frontend][4/N] Improve all pooling task | Add plugin pooling task #26973
• [Frontend][Doc][5/N] Improve all pooling task | Polish encode (pooling) api & Document. #25524

These PRs are mostly conflicting with each other, so combining them into a series would better inform reviewers about what happened. And what else needs to be done after that?

Purpose

Response compression if the client sends the 'accept-encoding' header: 'zstd, gzip'
Use Response compression to transmit binary files that don't need base64, cool. base64 is very inefficient
I wouldn't think of this hacky method before seeing this issue.

Thanks @uasan for this cool idea

Fix #27063

cc @christian-pinto @maxdebayser @DarkLight1337

Test Plan

tests/utils_/test_serial_utils.py
tests/entrypoints/pooling/openai/test_embedding.py
tests/entrypoints/pooling/openai/test_pooling.py

Test Result

pass

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @noooop.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[mergify]

Documentation preview: https://vllm--27066.org.readthedocs.build/en/27066/

[noooop]

/gemini review

[gemini-code-assist]

Code Review

This pull request introduces an excellent feature for improving performance by supporting binary embedding responses, including a new bytes encoding format and endianness customization. The refactoring of serialization logic into vllm/utils/serial_utils.py is a positive structural change.

However, my review has identified a critical data corruption bug in the new serialization logic specifically for bfloat16 tensors. The current implementation incorrectly reinterprets bfloat16 bit patterns as float16, which leads to corrupted data. This issue is not caught by the existing tests because they only verify round-trip consistency, where the symmetric corruption cancels itself out. I've provided a detailed comment with a code suggestion to fix this critical bug.

[noooop]

@DarkLight1337

Are there any more modifications needed for this PR?