Skip to content

Commit 663618c

Browse files
author
zhouyu5
committed
add comment
Signed-off-by: zhouyu5 <yu.zhou@intel.com>
1 parent e575e78 commit 663618c

File tree

1 file changed

+4
-0
lines changed

1 file changed

+4
-0
lines changed

vllm/model_executor/guided_decoding/outlines_logits_processors.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -91,6 +91,10 @@ def __call__(self, input_ids: List[int],
9191
allowed_tokens < scores.shape[-1])
9292
mask.index_fill_(0, allowed_tokens, 0)
9393
if current_platform.is_hpu():
94+
# Workaround for HPU bug where add_() raise RuntimeError:
95+
# synNodeCreateWithId failed for node: strided_insert
96+
# with synStatus 1 [Invalid argument], hopefully it will
97+
# be fixed in the future releases of the HPU runtime.
9498
scores = scores.add(mask)
9599
else:
96100
scores.add_(mask)

0 commit comments

Comments
 (0)