Skip to content

Commit 65c6409

Browse files
heheda12345gemini-code-assist[bot]
authored andcommitted
[Doc] Update the doc for log probs + prefix caching (vllm-project#23399)
Signed-off-by: Chen Zhang <zhangch99@outlook.com> Co-authored-by: gemini-code-assist[bot] <176961590+gemini-code-assist[bot]@users.noreply.github.com>
1 parent 0c5f12b commit 65c6409

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

docs/usage/v1_guide.md

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -166,7 +166,7 @@ Processed means the values after applying all processors, including temperature
166166

167167
##### Prompt Logprobs with Prefix Caching
168168

169-
Currently prompt logprobs are only supported when prefix caching is turned off via `--no-enable-prefix-caching`. In a future release, prompt logprobs will be compatible with prefix caching, but a recomputation will be triggered to recover the full prompt logprobs even upon a prefix cache hit. See details in [RFC #13414](gh-issue:13414).
169+
Logprobs are not cached. For a request requiring prompt logprobs, the engine will ignore the prefix cache and recompute the prefill of full prompt to generate the logprobs.
170170

171171
#### Deprecated Features
172172

0 commit comments

Comments
 (0)