[Core] Prevent side-channel attacks via cache salting

[dr75]

Context

Prefix caching in vLLM improves inference performance by reusing KV blocks across requests. However, this reuse introduces a potential privacy risk in shared environments, where an attacker could infer prompt reuse via timing side channels as demonstrated in Leaking Secrets from Prefix Caches.

To address this, we propose to isolate caches as described in an RFC: #16016

Suggested change

This PR implements the single barrier approach from the RFC by adding support for an optional cache_salt field in the request schema. When present, the salt is injected into the hash of the first block, ensuring that only requests with the same salt can share cached blocks. This effectively segments cache reuse by salt and protects against timing-based attacks.

{
  "messages": [
    {"role": "system", "content": "You are a helpful assistant."},
    {"role": "user", "content": "Here is a document with details about the world series: ..."},
    {"role": "user", "content": "Who won the world series in 2020?"}
  ],
  "cache_salt": "Z3V2bmV3aGxza3ZubGFoZ3Zud3V3ZWZ2bmd0b3V2bnZmc2xpZ3RoZ2x2aQ=="
}

The change is compatible with OpenAI requests as only an additional optional field is added. Users can still use the OpenAI client:

response = client.chat.completions.create(
    model=model,
    messages=messages,
    extra_body={
        "cache_salt": "Z3V2bmV3aGxza3ZubGFoZ3Zud3V3ZWZ2bmd0b3V2bnZmc2xpZ3RoZ2x2aQ==",
    },
)

The scope of cache sharing can be configured per request as needed, e.g., full single user protection or cache sharing within a group of users.

The change is in line with cache protection applied by other providers such as OpenAI and provides a solution that allows for higher flexibility allowing for more fine-grained configuration.

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[russellb]

I wonder how much this helps given that vLLM already initializes hashes with a random number that's different each time vLLM is executed. related: GHSA-rm76-4mrf-v9r8

[russellb]

I wonder how much this helps given that vLLM already initializes hashes with a random number that's different each time vLLM is executed. related: GHSA-rm76-4mrf-v9r8

sorry, I hadn't read the paper yet!

[russellb]

I think maintainers of the KV cache should review this, but I like this conceptually from a security perspective. Thank you!

[dr75]

This is related too: #15297

[dr75]

I think maintainers of the KV cache should review this, but I like this conceptually from a security perspective. Thank you!

Thanks! I am in touch with @comaniac for taking a look.

[dr75]

Both CI failures (both failing a bit differently) are in EntrypointsTest, in PEFTHelper.from_local_dir() reading a file so I guess unrelated.

[comaniac]

Otherwise LGTM. cc @WoosukKwon @ywang96

[comaniac]

IIUC, we only include cache salt in the first block of a prompt? Is any particular reason not to include it to all blocks?

[dr75]

It is propagated to all blocks via the hash of the previous block. So adding it to all would not improve it.

[dr75]

One thing I just realized @comaniac: I added it only to the V1 engine but that's problematic for V0. In addition to a comment in the docs I guess there should be some error for requests with salt to the V0 engine, otherwise users of V0 will provide a salt but don't get feedback that it is not used.

Any suggestion? Or do you consider V0 deprecated and its good enough to only have a comment in the docs?

[comaniac]

One thing I just realized @comaniac: I added it only to the V1 engine but that's problematic for V0. In addition to a comment in the docs I guess there should be some error for requests with salt to the V0 engine, otherwise users of V0 will provide a salt but don't get feedback that it is not used.

Any suggestion? Or do you consider V0 deprecated and its good enough to only have a comment in the docs?

Yeah definitely we should error out when v0 engine receives the cache salt.

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @dr75.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[dr75]

One thing I just realized @comaniac: I added it only to the V1 engine but that's problematic for V0. In addition to a comment in the docs I guess there should be some error for requests with salt to the V0 engine, otherwise users of V0 will provide a salt but don't get feedback that it is not used.
Any suggestion? Or do you consider V0 deprecated and its good enough to only have a comment in the docs?

Yeah definitely we should error out when v0 engine receives the cache salt.

Updated docs and return an error if the salt is used with V0.

[DarkLight1337]

Please fix pre-commit

[DarkLight1337]

And try to avoid force push. Each time you do it I have to read the whole PR again

[dr75]

And try to avoid force push. Each time you do it I have to read the whole PR again

Sorry, had to rebase. You prefer merging?

[DarkLight1337]

And try to avoid force push. Each time you do it I have to read the whole PR again

Sorry, had to rebase. You prefer merging?

Yeah, we will squash the PR at the end anyway

[dr75]

@DarkLight1337, seems good now. Shall we merge before it conflicts again?

[dr75]

Thanks for your support @DarkLight1337, @comaniac, @russellb !