Skip to content

Commit 2740c11

Browse files
committed
fix: walk around acc issue in eager mode for rope forward_hip (vllm-project#19)
1 parent bec8afa commit 2740c11

File tree

1 file changed

+2
-1
lines changed
  • vllm/model_executor/layers/rotary_embedding

1 file changed

+2
-1
lines changed

vllm/model_executor/layers/rotary_embedding/base.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -146,7 +146,8 @@ def forward_hip(
146146
else:
147147
# ops.rotary_embedding() is an in-place operation
148148
# that updates the query and key tensors.
149-
self.forward_cuda(positions, query, key)
149+
# FIXME: self.forward_cuda is not a in-place operation in eager mode.
150+
return self.forward_cuda(positions, query, key)
150151
return query, key
151152

152153
def forward_xpu(

0 commit comments

Comments
 (0)