[Attention] Move Backend enum into registry #25893

MatthewBonanni · 2025-09-29T15:21:06Z

Purpose

Creates vllm/attention/backends/registry.py, containing the _Backend enum. This PR follows #25489 and is a component of the larger attention backend refactor (now being split up), #24794. This file will contain more utility functions in the future.

Test Plan

CI should be sufficient

Test Result

Essential Elements of an Effective PR Description Checklist

The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
The test plan, such as providing test command.
The test results, such as pasting the results comparison before and after, or e2e results
(Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
(Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

gemini-code-assist

Code Review

This pull request is a good step in refactoring the attention backend by moving the _Backend enum to a centralized registry. The changes are consistent and improve the code structure. I've found one area for improvement in vllm/envs.py where a dynamic import is no longer necessary and can be simplified, which will enhance readability and maintainability.

vllm/envs.py

mergify · 2025-09-30T14:06:46Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @MatthewBonanni.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

NickLucche

Hey @MatthewBonanni thanks a lot for splitting this up, it's going to be much easier to review!
Evolving the registry into something more dynamic (for OOT plugins) makes sense with this structure imo.
Just one question, what is this PR going to enable as next step? Ie what's the first benefit the backend registry unlocks?

This change looks fairly safe to me, pinging @mgoin too.

NickLucche · 2025-09-30T15:03:59Z

vllm/attention/backends/registry.py

+import enum
+
+
+class _Backend(enum.Enum):


ok I suppose this is where @ILikeIneine would like to plug in a different attn backend.
I think we can add that later, although this "registry" here is a very manual one: usually registries should expose a way to get added to the registry, here it's just a plain enum for now

Yes, I see what you're saying. We could replace the enum with some other object, but the ergonomics of the enum are quite nice. Maybe we could construct an enum at import time?

checked out the other PR, I think it's fine as is and then we add the actual registration mechanism in the next PR.

For a future P: I think it might be nice to make this enum platform specific?

MatthewBonanni · 2025-09-30T15:53:19Z

@NickLucche thanks for your review! The next step is shown in #25900, in which backends can register themselves in the BACKEND_MAP, which is used to map from the enum to the object itself. This functionality will eventually be used in get_attn_backend_cls for the various platforms. This functionality was previously implemented in the attention test utils for use in those tests -- this registry will allow its use throughout the codebase

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

mergify · 2025-10-02T15:36:00Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @MatthewBonanni.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

LucasWilkinson

LGTM; one nit, thanks for doing this!

vllm/attention/backends/registry.py

LucasWilkinson · 2025-10-02T16:53:37Z

vllm/attention/backends/registry.py

+import enum
+
+
+class _Backend(enum.Enum):


For a future P: I think it might be nice to make this enum platform specific?

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

MatthewBonanni · 2025-10-02T17:01:30Z

@LucasWilkinson thanks for your review! Will definitely figure out how to make distinctions between platforms in the future. If some backends support multiple platforms (not sure if this is the case) it could be better to add a get_supported_platforms to the backends themselves? This would mesh better with my approach in #24794

vllm-project/vllm#25893 Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: yewentao256 <zhyanwentao@126.com>

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: Karan Goel <3261985+karan@users.noreply.github.com>

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

### What this PR does / why we need it? This is the step 1 of refactoring code to adapt with vllm main, and this pr aligned with vllm-project/vllm@17c540a 1. refactor deepseek to the latest code arch as of vllm-project/vllm@17c540a 2. bunches of fixes due to vllm changes - Fix `AscendScheduler` `__post_init__`, caused by vllm-project/vllm#25075 - Fix `AscendScheduler` init got an unexpected arg `block_size`, caused by vllm-project/vllm#26296 - Fix `KVCacheManager` `get_num_common_prefix_blocks` arg, caused by vllm-project/vllm#23485 - Fix `MLAAttention` import,caused by vllm-project/vllm#25103 - Fix `SharedFusedMoE` import, caused by vllm-project/vllm#26145 - Fix `LazyLoader` improt, caused by vllm-project/vllm#27022 - Fix `vllm.utils.swap_dict_values` improt, caused by vllm-project/vllm#26990 - Fix `Backend` enum import, caused by vllm-project/vllm#25893 - Fix `CompilationLevel` renaming to `CompilationMode` issue introduced by vllm-project/vllm#26355 - Fix fused_moe ops, caused by vllm-project/vllm#24097 - Fix bert model because of `inputs_embeds`, caused by vllm-project/vllm#25922 - Fix MRope because of `get_input_positions_tensor` to `get_mrope_input_positions`, caused by vllm-project/vllm#24172 - Fix `splitting_ops` changes introduced by vllm-project/vllm#25845 - Fix multi-modality changes introduced by vllm-project/vllm#16229 - Fix lora bias dropping issue introduced by vllm-project/vllm#25807 - Fix structured ouput break introduced by vllm-project/vllm#26737 ### Does this PR introduce _any_ user-facing change? ### How was this patch tested? CI passed with existing test. - vLLM version: v0.11.0rc3 - vLLM main: https://github.com/vllm-project/vllm/commit/v0.11.0 --------- Signed-off-by: MengqingCao <cmq0113@163.com> Signed-off-by: Icey <1790571317@qq.com> Co-authored-by: Icey <1790571317@qq.com>

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>

MatthewBonanni requested review from ApostaC, LucasWilkinson, NickLucche, WoosukKwon, bigPYJ1151, jikunshang, mgoin, sighingnow, tlrmchlsmth and yewentao256 as code owners September 29, 2025 15:21

mergify bot added qwen Related to Qwen models rocm Related to AMD ROCm speculative-decoding v1 tpu Related to Google TPUs kv-connector labels Sep 29, 2025

gemini-code-assist bot reviewed Sep 29, 2025

View reviewed changes

vllm/envs.py Show resolved Hide resolved

MatthewBonanni force-pushed the backend_registry branch from fcc6c0e to 16a1f40 Compare September 29, 2025 16:54

MatthewBonanni mentioned this pull request Sep 29, 2025

[Attention] Implement universal BACKEND_MAP #25900

Merged

5 tasks

MatthewBonanni changed the title ~~Move Backend enum into registry~~ [Attention] Move Backend enum into registry Sep 30, 2025

mergify bot added the needs-rebase label Sep 30, 2025

MatthewBonanni added 2 commits September 30, 2025 10:36

add registry

0001743

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

point to new registry

7c31e39

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

MatthewBonanni force-pushed the backend_registry branch from 16a1f40 to 7c31e39 Compare September 30, 2025 14:36

mergify bot removed the needs-rebase label Sep 30, 2025

NickLucche reviewed Sep 30, 2025

View reviewed changes

fix circular import (fix docs build)

a0e49d5

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

NickLucche added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 1, 2025

MatthewBonanni added 3 commits October 1, 2025 11:26

fix circular imports

82aa912

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

Merge branch 'main' into backend_registry

0c104b1

Merge branch 'main' into backend_registry

41f6adb

mergify bot added the needs-rebase label Oct 2, 2025

Merge branch 'main' into backend_registry

ef531ae

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

mergify bot removed the needs-rebase label Oct 2, 2025

LucasWilkinson approved these changes Oct 2, 2025

View reviewed changes

MatthewBonanni added 2 commits October 2, 2025 12:56

remove old comments

45eb2f5

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

remove deprecated backends

bc1d271

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

vllm-bot merged commit 2aaa423 into vllm-project:main Oct 3, 2025
56 of 58 checks passed

MatthewBonanni deleted the backend_registry branch October 3, 2025 04:34

xuechendi mentioned this pull request Oct 3, 2025

[FIX_FOR_VLLM_LATEST] fix issue brought by upstream PR #25893 vllm-project/vllm-gaudi#310

Merged

xuechendi added a commit to vllm-project/vllm-gaudi that referenced this pull request Oct 3, 2025

[FIX_FOR_VLLM_LATEST] fix issue brought by upstream PR #25893 (#310)

d73f260

vllm-project/vllm#25893 Signed-off-by: Chendi Xue <Chendi.Xue@intel.com>

Chenyaaang mentioned this pull request Oct 3, 2025

Fix CI error about attn backend cls vllm-project/tpu-inference#785

Merged

yewentao256 pushed a commit that referenced this pull request Oct 3, 2025

[Attention] Move Backend enum into registry (#25893)

2ea7d48

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: yewentao256 <zhyanwentao@126.com>

tomeras91 pushed a commit to tomeras91/vllm that referenced this pull request Oct 6, 2025

[Attention] Move Backend enum into registry (vllm-project#25893)

298e730

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: Tomer Asida <57313761+tomeras91@users.noreply.github.com>

karan pushed a commit to karan/vllm that referenced this pull request Oct 6, 2025

[Attention] Move Backend enum into registry (vllm-project#25893)

e236564

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: Karan Goel <3261985+karan@users.noreply.github.com>

southfreebird pushed a commit to southfreebird/vllm that referenced this pull request Oct 7, 2025

[Attention] Move Backend enum into registry (vllm-project#25893)

030c42c

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

leo-pony mentioned this pull request Oct 9, 2025

[CI] fix the vllm run-npu-test.sh break: cannot import _Backend from vllm.platforms vllm-project/vllm-ascend#3337

Closed

xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 10, 2025

[Attention] Move Backend enum into registry (vllm-project#25893)

a4cc3d2

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>

This was referenced Oct 16, 2025

[CI] Upgrade vllm to newest commit vllm-project/vllm-ascend#3423

Closed

[CI] Upgrade vllm to 0.11.1 vllm-project/vllm-ascend#3499

Closed

lywa1998 pushed a commit to lywa1998/vllm that referenced this pull request Oct 20, 2025

[Attention] Move Backend enum into registry (vllm-project#25893)

43ab4d0

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

MengqingCao mentioned this pull request Oct 22, 2025

[1/N][Refactor] Refactor code to adapt with vllm main vllm-project/vllm-ascend#3612

Merged

alhridoy pushed a commit to alhridoy/vllm that referenced this pull request Oct 24, 2025

[Attention] Move Backend enum into registry (vllm-project#25893)

8598190

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com>

xuebwang-amd pushed a commit to xuebwang-amd/vllm that referenced this pull request Oct 24, 2025

[Attention] Move Backend enum into registry (vllm-project#25893)

3143da6

Signed-off-by: Matthew Bonanni <mbonanni@redhat.com> Signed-off-by: xuebwang-amd <xuebwang@amd.com>

Uh oh!

[Attention] Move Backend enum into registry #25893

[Attention] Move Backend enum into registry #25893

Uh oh!

Conversation

MatthewBonanni commented Sep 29, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Purpose

Test Plan

Test Result

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

mergify bot commented Sep 30, 2025

Uh oh!

NickLucche left a comment

Choose a reason for hiding this comment

Uh oh!

NickLucche Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

MatthewBonanni Sep 30, 2025

Choose a reason for hiding this comment

Uh oh!

NickLucche Oct 1, 2025

Choose a reason for hiding this comment

Uh oh!

LucasWilkinson Oct 2, 2025

Choose a reason for hiding this comment

Uh oh!

MatthewBonanni commented Sep 30, 2025

Uh oh!

mergify bot commented Oct 2, 2025

Uh oh!

LucasWilkinson left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

LucasWilkinson Oct 2, 2025

Choose a reason for hiding this comment

Uh oh!

MatthewBonanni commented Oct 2, 2025

Uh oh!

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

MatthewBonanni commented Sep 29, 2025 •

edited by github-actions bot

Loading