Skip to content

Conversation

@elvischenv
Copy link
Contributor

@elvischenv elvischenv commented Jul 30, 2025

Essential Elements of an Effective PR Description Checklist

  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.

Purpose

The motivation is we cannot find the compilation_config.pass_config option from non-default args, that's confusing whether our config is working or not.

before

INFO 07-30 04:45:26 [utils.py:326] non-default args: {'model_tag': 'nvidia/Llama-4-Scout-17B-16E-Instruct-FP8', ..., 'compilation_config': {"level":3, ...,"full_cuda_graph":false,"max_capture_size":null,"local_cache_dir":null}, 'disable_log_requests': True}

after

INFO 07-30 05:57:17 [utils.py:326] non-default args: {'model_tag': 'nvidia/Llama-4-Scout-17B-16E-Instruct-FP8', ..., 'compilation_config': {"level":3, ...,"full_cuda_graph":false,"pass_config":{"enable_fusion":false,"enable_attn_fusion":false,"enable_noop":false,"enable_sequence_parallelism":false,"enable_async_tp":false,"enable_fi_allreduce_fusion":true,"fi_allreduce_fusion_max_token_num":1024},"max_capture_size":null,"local_cache_dir":null}, 'disable_log_requests': True}

Test Plan

Test Result

(Optional) Documentation Update

Copy link
Contributor

@gemini-code-assist gemini-code-assist bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request correctly removes pass_config from the exclusion list in CompilationConfig.__repr__, which will now include it in the logged non-default arguments. This is a useful change for debugging and verifying compilation configurations. The minor docstring typo fix is also a good cleanup. The changes are correct and well-motivated.

@github-actions
Copy link

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

@hmellor
Copy link
Member

hmellor commented Jul 31, 2025

cc @ProExpertProg do we want to include pass_config in the repr?

@nvpohanh
Copy link
Contributor

nvpohanh commented Aug 1, 2025

To add some context: the pass_config has large impact on performance numbers. Currently, we do not know if a non-default pass config is taking effects or not by looking at the logs. If you have other ways to check that please let us know. thanks!

@ProExpertProg
Copy link
Collaborator

I think originally @youkaichao removed pass config from repr because it's very verbose. But I agree with new passes that heavily affect the FX graph and performance, we should likely include it

@ProExpertProg
Copy link
Collaborator

Perhaps we could either only display the true values or the non-defaults. Not sure what's best here

@nvpohanh
Copy link
Contributor

nvpohanh commented Aug 4, 2025

@elvischenv Is it possible to only show the "non-default" fields in pass_config?

Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
@elvischenv elvischenv force-pushed the elvischenv/remove-pass-config branch from 3abae86 to adf765d Compare August 4, 2025 06:21
@elvischenv
Copy link
Contributor Author

elvischenv commented Aug 4, 2025

Update to only show the non-default flags for pass_config:

before:

INFO 08-04 05:39:52 [utils.py:326] non-default args: {
  ...,
  'compilation_config': {
    "level":3,
    "debug_dump_path":"",
    "cache_dir":"",
    "backend":"",
    "custom_ops":["+rms_norm"],
    "splitting_ops":[],
    "use_inductor":true,
    "compile_sizes":null,
    "inductor_compile_config":{"enable_auto_functionalized_v2":false},
    "inductor_passes":{},
    "use_cudagraph":true,
    "cudagraph_num_of_warmups":0,
    "cudagraph_capture_sizes":null,
    "cudagraph_copy_inputs":false,
    "full_cuda_graph":false,
    "max_capture_size":null,
    "local_cache_dir":null
  }
}

PR:

INFO 08-04 06:18:11 [utils.py:326] non-default args: {
  ...,
  'compilation_config': {
    "level":3,
    "debug_dump_path":"",
    "cache_dir":"",
    "backend":"",
    "custom_ops":["+rms_norm"],
    "splitting_ops":[],
    "use_inductor":true,
    "compile_sizes":null,
    "inductor_compile_config":{"enable_auto_functionalized_v2":false},
    "inductor_passes":{},
    "use_cudagraph":true,
    "cudagraph_num_of_warmups":0,
    "cudagraph_capture_sizes":null,
    "cudagraph_copy_inputs":false,
    "full_cuda_graph":false,
    "pass_config":{"enable_fi_allreduce_fusion":true},
    "max_capture_size":null,
    "local_cache_dir":null
  }
}

@nvpohanh
Copy link
Contributor

nvpohanh commented Aug 4, 2025

@ProExpertProg FYI, @elvischenv has updated it to only print non-defaults. Does it look okay to you? thanks!

@hmellor hmellor enabled auto-merge (squash) August 4, 2025 10:40
@github-actions github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Aug 4, 2025
@vllm-bot vllm-bot merged commit 7b455cf into vllm-project:main Aug 5, 2025
45 of 49 checks passed
npanpaliya pushed a commit to odh-on-pz/vllm-upstream that referenced this pull request Aug 6, 2025
…llm-project#21911)

Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
@elvischenv elvischenv deleted the elvischenv/remove-pass-config branch August 7, 2025 00:39
myselvess pushed a commit to myselvess/vllm that referenced this pull request Aug 7, 2025
…llm-project#21911)

Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
jinzhen-lin pushed a commit to jinzhen-lin/vllm that referenced this pull request Aug 9, 2025
…llm-project#21911)

Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Signed-off-by: Jinzhen Lin <linjinzhen@hotmail.com>
noamgat pushed a commit to noamgat/vllm that referenced this pull request Aug 9, 2025
…llm-project#21911)

Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Signed-off-by: Noam Gat <noamgat@gmail.com>
paulpak58 pushed a commit to paulpak58/vllm that referenced this pull request Aug 13, 2025
…llm-project#21911)

Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Signed-off-by: Paul Pak <paulpak58@gmail.com>
diegocastanibm pushed a commit to diegocastanibm/vllm that referenced this pull request Aug 15, 2025
…llm-project#21911)

Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Signed-off-by: Diego-Castan <diego.castan@ibm.com>
epwalsh pushed a commit to epwalsh/vllm that referenced this pull request Aug 28, 2025
…llm-project#21911)

Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
xiao-llm pushed a commit to xiao-llm/vllm that referenced this pull request Aug 28, 2025
…llm-project#21911)

Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Signed-off-by: Xiao Yu <xiao.yu@amd.com>
zhewenl pushed a commit to zhewenl/vllm that referenced this pull request Aug 28, 2025
…llm-project#21911)

Signed-off-by: elvischenv <219235043+elvischenv@users.noreply.github.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ready ONLY add when PR is ready to merge/full CI is needed

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants