Skip to content

Commit d236ccf

Browse files
tstescoTTskhorasganiTT
authored andcommitted
fixing vllm-project#31 by converting SamplerOutput output_token to int in TTModelRunner::_make_sampler_output as expected by vLLM downstream
1 parent 35dd45f commit d236ccf

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/worker/tt_model_runner.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -351,7 +351,7 @@ def _make_sampler_output(
351351
zero_logprob = Logprob(0.0)
352352
sampler_outputs = []
353353
for batch_idx, seq_id in enumerate(seq_groups):
354-
next_token_id = next_token_ids[batch_idx]
354+
next_token_id = int(next_token_ids[batch_idx])
355355
seq_outputs = [SequenceOutput(seq_id, next_token_id,
356356
{next_token_id: zero_logprob})]
357357
sampler_outputs.append(

0 commit comments

Comments
 (0)