Skip to content

Commit 6e1e1ca

Browse files
wenhuach21amitm02
authored andcommitted
improve the robustness of parsing vlms config in AutoRound (vllm-project#18894)
Signed-off-by: wenhuach21 <wenhua.cheng@intel.com> Signed-off-by: amit <amit.man@gmail.com>
1 parent 52f61db commit 6e1e1ca

File tree

1 file changed

+3
-2
lines changed

1 file changed

+3
-2
lines changed

vllm/model_executor/layers/quantization/auto_round.py

Lines changed: 3 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -116,8 +116,9 @@ def get_layer_config(self, layer, layer_name: str):
116116

117117
quantized = True
118118
if self.block_name_to_quantize:
119-
quantized = any(name in layer_name
120-
for name in self.block_name_to_quantize)
119+
quantized = any(
120+
layer_name.startswith(name)
121+
for name in self.block_name_to_quantize)
121122
elif isinstance(layer, ParallelLMHead):
122123
quantized = False
123124

0 commit comments

Comments
 (0)