Skip to content

Commit ce4e43f

Browse files
Jialinalhridoy
authored andcommitted
[Perf][Easy] Early stop in request_block_hasher (vllm-project#26112)
Signed-off-by: Jialin Ouyang <Jialin.Ouyang@gmail.com>
1 parent e912450 commit ce4e43f

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

vllm/v1/core/kv_cache_utils.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -588,6 +588,10 @@ def request_block_hasher(request: Request) -> list[BlockHash]:
588588
start_token_idx = len(request.block_hashes) * block_size
589589
num_tokens = request.num_tokens
590590

591+
if start_token_idx + block_size > num_tokens:
592+
# Early stop when there no new full blocks created.
593+
return []
594+
591595
curr_mm_idx = 0
592596
if start_token_idx > 0:
593597
# Set curr_mm_idx = -1 to indicate the last mm input.

0 commit comments

Comments
 (0)