Skip to content

Commit c091c0a

Browse files
authored
Improve validation of TP in Transformers backend (#15540)
Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>
1 parent 1aa162e commit c091c0a

File tree

1 file changed

+4
-1
lines changed

1 file changed

+4
-1
lines changed

vllm/model_executor/models/transformers.py

Lines changed: 4 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -229,7 +229,10 @@ def tensor_parallel(self):
229229
Apply the model's tensor parallelization plan.
230230
Currently only supports linear layers.
231231
"""
232-
if self.tp_size > 1 and self.config.base_model_tp_plan is None:
232+
if not self.model.supports_tp_plan:
233+
if self.tp_size <= 1:
234+
return
235+
233236
raise ValueError(
234237
f"{type(self.model)} does not support tensor parallel yet!")
235238

0 commit comments

Comments
 (0)