Skip to content

Commit 2c685e3

Browse files
authored
[Bugfix] Correct method call for _set_cos_sin_cache (#774)
This change ensures proper functionality for longer sequences by correctly invoking the _set_cos_sin_cache method with self as the first argument. For example, with DeepSeek R1, if this change isn't made, the program will crash when the input sequence exceeds 4096. Signed-off-by: Jade Zheng <zheng.shoujian@outlook.com>
1 parent 5301649 commit 2c685e3

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm_ascend/ops/rotary_embedding.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -82,7 +82,7 @@ def native_rope_deepseek_forward(self,
8282
offsets: Optional[torch.Tensor] = None,
8383
max_seq_len: Optional[int] = None):
8484
if max_seq_len is not None and max_seq_len > self.max_seq_len:
85-
self._set_cos_sin_cache(max_seq_len, query.device, query.dtype)
85+
_set_cos_sin_cache(self, max_seq_len, query.device, query.dtype)
8686
if len(key.shape) == 2:
8787
key = key[:, None, :]
8888
# Note: we implement the non neox_style method with shuffle the last dim and neox style

0 commit comments

Comments
 (0)