Skip to content

Conversation

@huydhn
Copy link
Contributor

@huydhn huydhn commented Jun 28, 2025

There is a bug in the script where I wrongly used models variable as both the input list of target models and to de-dup the list of selected models.

Testing

https://github.com/pytorch/pytorch-integration-testing/actions/runs/15940471919/job/44967831823

python .github/scripts/generate_vllm_benchmark_matrix.py --benchmark-configs-dir vllm-benchmarks/benchmarks --models "meta-llama/Llama-4-Maverick-17B-128E-Instruct-FP8" --gpus h100

returns only one job now

::set-output name=benchmark_matrix::{'include': [{'runner': 'linux.aws.h100.8', 'models': 'meta-llama/llama-4-maverick-17b-128e-instruct-fp8'}]}

huydhn added 12 commits June 19, 2025 17:36
Signed-off-by: Huy Do <huydhn@gmail.com>
Signed-off-by: Huy Do <huydhn@gmail.com>
Signed-off-by: Huy Do <huydhn@gmail.com>
Signed-off-by: Huy Do <huydhn@gmail.com>
Signed-off-by: Huy Do <huydhn@gmail.com>
Signed-off-by: Huy Do <huydhn@gmail.com>
Signed-off-by: Huy Do <huydhn@gmail.com>
Signed-off-by: Huy Do <huydhn@gmail.com>
Signed-off-by: Huy Do <huydhn@gmail.com>
Signed-off-by: Huy Do <huydhn@gmail.com>
Signed-off-by: Huy Do <huydhn@gmail.com>
Signed-off-by: Huy Do <huydhn@gmail.com>
Copy link
Contributor

@yangw-dev yangw-dev left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM!

@huydhn huydhn merged commit 55172b0 into main Jul 2, 2025
6 of 9 checks passed
@huydhn huydhn deleted the fix-model-selection branch July 9, 2025 07:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants