Skip to content

Commit 3c02a28

Browse files
authored
Add 256 as possible option within block-size arg (#1279)
Bringing back option of 256 as possible block-size arg value, that has been lost in some of the last rebases. It has been first added via #971 The options of arguments are now defined by unpacking predefined type hints ![image](https://github.com/user-attachments/assets/f1c0429b-6449-44a5-b0e8-326f465590ab) https://github.com/HabanaAI/vllm-fork/blob/habana_main/vllm/engine/arg_utils.py#L611
1 parent 90f1ba8 commit 3c02a28

File tree

1 file changed

+1
-1
lines changed

1 file changed

+1
-1
lines changed

vllm/config.py

Lines changed: 1 addition & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -1325,7 +1325,7 @@ def matryoshka_dimensions(self):
13251325
return getattr(self.hf_config, "matryoshka_dimensions", None)
13261326

13271327

1328-
BlockSize = Literal[1, 8, 16, 32, 64, 128]
1328+
BlockSize = Literal[1, 8, 16, 32, 64, 128, 256]
13291329
CacheDType = Literal["auto", "fp8", "fp8_e4m3", "fp8_e5m2", "fp8_inc"]
13301330
PrefixCachingHashAlgo = Literal["builtin", "sha256"]
13311331

0 commit comments

Comments
 (0)