Skip to content

Commit 487e344

Browse files
committed
[Model] Integrate PARD into vLLM pre-commit
Signed-off-by: root <anzihao_hh@126.com> Signed-off-by: <anzihao_hh@126.com>
1 parent 8bddd03 commit 487e344

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/spec_decode/multi_step_worker.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -136,7 +136,7 @@ def pard_infer(self, expanded_request: ExecuteModelRequest,
136136
# prepare recompute kv token
137137
# update seq_group_metadata_list
138138
mask_token_id = self.pard_token
139-
request_by_id = {}
139+
request_by_id: Dict[str, List[SequenceGroupMetadata]] = {}
140140
for tmp in expanded_request.seq_group_metadata_list:
141141
name = tmp.request_id
142142
request_by_id[name] = request_by_id.get(name, [])

0 commit comments

Comments
 (0)