[Feature]: Support Prefix Caching for Hidden States (Pooling Endpoint)

[Risc-lt]

🚀 The feature, motivation and pitch

I would like the /pooling endpoint to support prefix caching for hidden states.

Background

The /pooling endpoint is designed to extract hidden states/embeddings by performing a full prefill pass over all input tokens. However, it currently doesn't support prefix caching - every request recomputes all tokens from scratch, even for repeated prefixes.

Feature Request:

Enable prefix caching for the /pooling endpoint, so that:

• Hidden states for cache-hit tokens are retrieved from cache (not recomputed)
• Only new/uncached tokens need computation
• The complete hidden states (cached + newly computed) are returned

Why This Matters:

Many applications process the same prefixes repeatedly (system prompts, instruction templates, etc.):

• Without hidden state caching: every /pooling request recomputes the entire sequence
• With hidden state caching: reuse cached hidden states -> only compute new tokens -> much better throughput

Alternatives

Currently, the only option is to use /pooling without prefix caching, which results in high latency for repeated prefixes.

Additional context

Related issues:

• [RFC]: Hidden states processor #12249
• [Feature]: Support Multiple Tasks Per Model #11905

This feature would require caching hidden states alongside KV cache, sharing the same prefix matching logic and eviction policy.

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the [documentation page](https://docs.vllm.ai/en/latest/), which can answer lots of frequently asked questions.

[noooop]

Theoretically, vllm supports all pooling + Prefix Caching

However, the logic for enabling Prefix Caching seems to be coupled with the logic for enabling chunked_prefill, and enabling chunked_prefill is coupled with the encode-decode model.

(╯‵□′)╯︵┻━┻

Improve all pooling task series PRs will solve these small issues, and polish APIs

I think this issue can be resolved after landing the two PRs below.

• [Model][6/N] Improve all pooling task | Improve enable chunked_prefill & prefix_caching logic. #26623
• [Model][7/N] Improve all pooling task | Deprecation as_reward_model. Extract hidden states prefer using new multi-vector retrieval API #26686