Skip to content

Commit 5b869d9

Browse files
acisseJZhongfacebook-github-bot
authored andcommitted
test llama4x HF with config parser (#25912)
Summary: test llama4x with shardist config parser Test Plan: buck test //vllm/fb/tests\:test_llama4x_hf -- --print-passing-details https://www.internalfb.com/intern/testinfra/testconsole/testrun/1970325152656551/ buck test 'fbcode//mode/opt' fbcode//vllm/fb/plugins/tests:test_shardist_loader -- --exact 'vllm/fb/plugins/tests:test_shardist_loader - test_shardist_model_loading (vllm.fb.plugins.tests.test_shardist_loader.TestShardistLoader)' --run-disabled https://www.internalfb.com/intern/testinfra/testconsole/testrun/844425379106729/ Differential Revision: D83405333
1 parent 6a113d9 commit 5b869d9

File tree

2 files changed

+4
-0
lines changed

2 files changed

+4
-0
lines changed

vllm/config/model.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -1334,11 +1334,13 @@ def try_get_generation_config(self) -> dict[str, Any]:
13341334
self.hf_config_path or self.model,
13351335
trust_remote_code=self.trust_remote_code,
13361336
revision=self.revision,
1337+
config_format=self.config_format,
13371338
)
13381339
else:
13391340
config = try_get_generation_config(
13401341
self.generation_config,
13411342
trust_remote_code=self.trust_remote_code,
1343+
config_format=self.config_format,
13421344
)
13431345

13441346
if config is None:

vllm/transformers_utils/config.py

Lines changed: 2 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -949,6 +949,7 @@ def try_get_generation_config(
949949
model: str,
950950
trust_remote_code: bool,
951951
revision: Optional[str] = None,
952+
config_format: Union[str, ConfigFormat] = "auto",
952953
) -> Optional[GenerationConfig]:
953954
try:
954955
return GenerationConfig.from_pretrained(
@@ -961,6 +962,7 @@ def try_get_generation_config(
961962
model,
962963
trust_remote_code=trust_remote_code,
963964
revision=revision,
965+
config_format=config_format,
964966
)
965967
return GenerationConfig.from_model_config(config)
966968
except OSError: # Not found

0 commit comments

Comments
 (0)