Skip to content

Commit 60e419c

Browse files
authored
[Misc] cache result of disable_inplace (#26666)
Signed-off-by: Bill Nell <bnell@redhat.com>
1 parent 7ef6052 commit 60e419c

File tree

1 file changed

+2
-0
lines changed
  • vllm/model_executor/layers/fused_moe

1 file changed

+2
-0
lines changed

vllm/model_executor/layers/fused_moe/utils.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1,5 +1,6 @@
11
# SPDX-License-Identifier: Apache-2.0
22
# SPDX-FileCopyrightText: Copyright contributors to the vLLM project
3+
import functools
34
from math import prod
45

56
import torch
@@ -325,5 +326,6 @@ def activation_without_mul(activation: str) -> str:
325326
# Torch custom ops can't deal with outputs aliasing inputs so we need to
326327
# disable inplace for torch >= 2.9.
327328
# See https://github.com/vllm-project/vllm/issues/26378
329+
@functools.cache
328330
def disable_inplace() -> bool:
329331
return is_torch_equal_or_newer("2.9")

0 commit comments

Comments
 (0)