Skip to content

Commit bfbad71

Browse files
authored
Fix upstream PR 22668 that added additional arg to is_kv_cache_dtype_supported (#96)
Fixes vllm-project/vllm#22668 - we need to take one more arg. Signed-off-by: Marcin Swiniarski <mswiniarski@habana.ai>
1 parent b8217f6 commit bfbad71

File tree

1 file changed

+2
-1
lines changed

1 file changed

+2
-1
lines changed

vllm_gaudi/platform.py

Lines changed: 2 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -154,7 +154,8 @@ def set_torch_compile(cls) -> None:
154154
os.environ['PT_HPU_ENABLE_LAZY_COLLECTIVES'] = 'true'
155155

156156
@classmethod
157-
def is_kv_cache_dtype_supported(cls, kv_cache_dtype: str) -> bool:
157+
def is_kv_cache_dtype_supported(cls, kv_cache_dtype: str,
158+
model_config: ModelConfig) -> bool:
158159
return kv_cache_dtype == "fp8_inc"
159160

160161
@classmethod

0 commit comments

Comments
 (0)