[FrontEnd] UNREVERT CompilationConfig overhaul (#20283): deprecate use_inductor in favor of backend, simplify custom_ops #26502

morrison-turnansky · 2025-10-09T15:48:09Z

Original PR: #26113
Revert of original PR: #26472

Purpose

Test Plan

Test Result

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
(Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

vllm/config/vllm.py

morrison-turnansky · 2025-10-09T16:03:56Z

original PR: #26113

morrison-turnansky · 2025-10-09T16:05:01Z

revert PR: #26472

hmellor · 2025-10-09T16:38:52Z

vllm/config/vllm.py

-                self.compilation_config.level > 0
-                and self.compilation_config.backend != "eager"
-            ):
+            if self.compilation_config.backend == "inductor":


Can we add a comment to say why we disable custom ops with inductor?

I reverted this to the logic from main.

mgoin · 2025-10-09T17:11:53Z

Can you summarize what are the changes relative to the original PR to fix the issues?

tests/compile/test_basic_correctness.py

ProExpertProg · 2025-10-09T20:25:40Z

cc @Chenyaaang

Chenyaaang · 2025-10-09T21:15:51Z

cc @Chenyaaang

May I ask what's the difference with the original PR? I believe it will still fail for TPU platform.

ProExpertProg · 2025-10-10T00:59:05Z

@Chenyaaang TPU-specific fixes have not been added yet, except the custom ops on by default only if backend == "inductor" (before it was if backend != "eager")

Chenyaaang · 2025-10-10T01:39:20Z

@Chenyaaang TPU-specific fixes have not been added yet, except the custom ops on by default only if backend == "inductor" (before it was if backend != "eager")

Got it, will it be included in this pr?

mergify · 2025-10-13T12:43:25Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @morrison-turnansky.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

ProExpertProg · 2025-10-13T18:41:13Z

Manually kicked off some blackwell and distributed tests just from abundance of caution. Seems like the rest of the tests are all passing!

vllm/config/compilation.py

ProExpertProg

Looks good apart from the one level thing. If you update, you might as well do the other nits 😅

tests/compile/test_basic_correctness.py

vllm/config/compilation.py

tests/model_executor/test_enabled_custom_ops.py

vllm/config/vllm.py

vllm/model_executor/custom_op.py

Signed-off-by: morrison-turnansky <mturnans@redhat.com>

Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Signed-off-by: Morrison Turnansky <mturnans@redhat.com>

Signed-off-by: morrison-turnansky <mturnans@redhat.com>

vllm/config/compilation.py

Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

…deprecate use_inductor in favor of backend, simplify custom_ops (vllm-project#26502) Signed-off-by: morrison-turnansky <mturnans@redhat.com> Signed-off-by: Morrison Turnansky <mturnans@redhat.com> Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com> Signed-off-by: 1994 <1994@users.noreply.github.com>

…deprecate use_inductor in favor of backend, simplify custom_ops (vllm-project#26502) Signed-off-by: morrison-turnansky <mturnans@redhat.com> Signed-off-by: Morrison Turnansky <mturnans@redhat.com> Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com> Signed-off-by: Dhruvil Bhatt <bhattdbh@amazon.com>

…deprecate use_inductor in favor of backend, simplify custom_ops (vllm-project#26502) Signed-off-by: morrison-turnansky <mturnans@redhat.com> Signed-off-by: Morrison Turnansky <mturnans@redhat.com> Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com> Signed-off-by: bbartels <benjamin@bartels.dev>

…deprecate use_inductor in favor of backend, simplify custom_ops (vllm-project#26502) Signed-off-by: morrison-turnansky <mturnans@redhat.com> Signed-off-by: Morrison Turnansky <mturnans@redhat.com> Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com>

…deprecate use_inductor in favor of backend, simplify custom_ops (vllm-project#26502) Signed-off-by: morrison-turnansky <mturnans@redhat.com> Signed-off-by: Morrison Turnansky <mturnans@redhat.com> Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>

…deprecate use_inductor in favor of backend, simplify custom_ops (vllm-project#26502) Signed-off-by: morrison-turnansky <mturnans@redhat.com> Signed-off-by: Morrison Turnansky <mturnans@redhat.com> Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Co-authored-by: Jiangyun Zhu <riverclouds.zhu@qq.com> Signed-off-by: 0xrushi <6279035+0xrushi@users.noreply.github.com>

morrison-turnansky requested review from ProExpertProg, WoosukKwon, bigPYJ1151, hmellor, houseroad, mgoin, robertgshaw2-redhat, simon-mo, tlrmchlsmth, yewentao256, youkaichao and zou3519 as code owners October 9, 2025 15:48

mergify bot added the llama Related to Llama models label Oct 9, 2025

mergify bot mentioned this pull request Oct 9, 2025

[Frontend] CompilationConfig overhaul (#20283): deprecate use_inductor in favor of backend, simplify custom_ops #26113

Merged

5 tasks

chatgpt-codex-connector bot reviewed Oct 9, 2025

View reviewed changes

vllm/config/vllm.py Show resolved Hide resolved

hmellor reviewed Oct 9, 2025

View reviewed changes

morrison-turnansky force-pushed the issue-20283-compilation-config branch from 5d35d77 to ce2132c Compare October 9, 2025 19:26

ProExpertProg reviewed Oct 9, 2025

View reviewed changes

tests/compile/test_basic_correctness.py Outdated Show resolved Hide resolved

ProExpertProg added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 9, 2025

hmellor mentioned this pull request Oct 9, 2025

[CI] Fix mypy for vllm/attention and vllm/compilation #26482

Open

ProExpertProg mentioned this pull request Oct 11, 2025

[compile] Fix inductor partition config #26645

Merged

morrison-turnansky force-pushed the issue-20283-compilation-config branch from ca21ddb to 849a150 Compare October 13, 2025 12:42

zou3519 reviewed Oct 13, 2025

View reviewed changes

vllm/config/compilation.py Show resolved Hide resolved

vllm/config/compilation.py Show resolved Hide resolved

vllm/config/compilation.py Show resolved Hide resolved

ProExpertProg approved these changes Oct 13, 2025

View reviewed changes

zou3519 approved these changes Oct 13, 2025

View reviewed changes

morrison-turnansky and others added 6 commits October 13, 2025 20:08

adjusted for comments

e438c43

Signed-off-by: morrison-turnansky <mturnans@redhat.com>

Update vllm/model_executor/custom_op.py

96e2c1e

Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Signed-off-by: Morrison Turnansky <mturnans@redhat.com>

Update vllm/config/vllm.py

0f8e420

Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Signed-off-by: Morrison Turnansky <mturnans@redhat.com>

Update vllm/config/compilation.py

975c909

Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Signed-off-by: Morrison Turnansky <mturnans@redhat.com>

Update tests/compile/test_basic_correctness.py

65bc52a

Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Signed-off-by: Morrison Turnansky <mturnans@redhat.com>

Update tests/compile/test_basic_correctness.py

51d4aa4

Co-authored-by: Luka Govedič <ProExpertProg@users.noreply.github.com> Signed-off-by: Morrison Turnansky <mturnans@redhat.com>

ProExpertProg removed the ready ONLY add when PR is ready to merge/full CI is needed label Oct 13, 2025

reviewer comments

5b5455c

Signed-off-by: morrison-turnansky <mturnans@redhat.com>

ProExpertProg approved these changes Oct 13, 2025

View reviewed changes

vllm/config/compilation.py Outdated Show resolved Hide resolved

Add version number to deprecation warning

c25bafb

Signed-off-by: Luka Govedič <ProExpertProg@users.noreply.github.com>

ProExpertProg enabled auto-merge (squash) October 13, 2025 20:54

github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 13, 2025

ProExpertProg merged commit e3fdb62 into vllm-project:main Oct 13, 2025
49 checks passed

ProExpertProg added this to the vllm==v0.12.0/torch==2.9.0 compilation improvements milestone Oct 13, 2025

ProExpertProg mentioned this pull request Oct 15, 2025

[Frontend][torch.compile] CompilationConfig Overhaul (#20283): name change compilation level to compilation mode, deprecation compilation level #26355

Merged

5 tasks

ProExpertProg mentioned this pull request Oct 17, 2025

[TPU] Fix tpu torch compile error #27049

Open

ProExpertProg linked an issue Nov 3, 2025 that may be closed by this pull request

[RFC][UX][torch.compile][CUDAGraph]: Overhaul CompilationConfig and improve CLI -O<n> #20283

Open

1 task

Uh oh!

[FrontEnd] UNREVERT CompilationConfig overhaul (#20283): deprecate use_inductor in favor of backend, simplify custom_ops #26502

[FrontEnd] UNREVERT CompilationConfig overhaul (#20283): deprecate use_inductor in favor of backend, simplify custom_ops #26502

Uh oh!

Conversation

morrison-turnansky commented Oct 9, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

morrison-turnansky commented Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

morrison-turnansky commented Oct 9, 2025

Uh oh!

hmellor Oct 9, 2025

Choose a reason for hiding this comment

Uh oh!

morrison-turnansky Oct 13, 2025

Choose a reason for hiding this comment

Uh oh!

mgoin commented Oct 9, 2025

Uh oh!

Uh oh!

ProExpertProg commented Oct 9, 2025

Uh oh!

Chenyaaang commented Oct 9, 2025

Uh oh!

ProExpertProg commented Oct 10, 2025

Uh oh!

Chenyaaang commented Oct 10, 2025

Uh oh!

mergify bot commented Oct 13, 2025

Uh oh!

ProExpertProg commented Oct 13, 2025

Uh oh!

Uh oh!

Uh oh!

Uh oh!

ProExpertProg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

6 participants

morrison-turnansky commented Oct 9, 2025 •

edited by github-actions bot

Loading

morrison-turnansky commented Oct 9, 2025 •

edited

Loading