[CI] Fix mypy for `vllm/attention` and `vllm/compilation` #26482

hmellor · 2025-10-09T11:49:06Z

This change brings us closer to not needing a custom mypy check. Part of #26533

This change brings us closer to not needing a custom `mypy` check. Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

chatgpt-codex-connector

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

Open a pull request for review
Mark a draft as ready
Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

vllm/attention/selector.py

vllm/config/vllm.py

gemini-code-assist

Code Review

This pull request aims to fix mypy issues to enable stricter type checking. While most changes correctly add or refine type hints, I've identified a few regressions. Specifically, a logic change in vllm/attention/selector.py removes the fallback to a default attention backend, which could break existing setups. Additionally, the logic for determining the default compilation level in vllm/config/vllm.py appears to be inverted with respect to the enforce_eager flag. Lastly, a type hint in vllm/v1/attention/backends/utils.py was changed incorrectly, which could lead to runtime errors. These issues are of high and critical severity and should be addressed.

vllm/config/vllm.py

vllm/v1/attention/backends/utils.py

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

vllm/config/device.py

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

hmellor · 2025-10-09T12:28:15Z

vllm/config/vllm.py

        else:
            assert self.compilation_config.level >= CompilationLevel.NO_COMPILATION
            assert self.compilation_config.level <= CompilationLevel.PIECEWISE
-            assert self.compilation_config.level <= 3


No longer needed because level is Field(default=None, ge=0, le=3) so Pydantic will catch it

vllm/config/compilation.py

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

mergify · 2025-10-09T12:46:02Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @hmellor.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

ProExpertProg

Thanks for the PR! Is the Field contract that fields will not be None after __post_init__?

vllm/config/compilation.py

vllm/config/vllm.py

vllm/config/compilation.py

hmellor · 2025-10-09T21:16:20Z

I believe the conflicts come from the revert of #26113, waiting for #26502 to merge so that they hopefully disappear.

hmellor · 2025-10-09T21:23:58Z

Is the Field contract that fields will not be None after post_init?

This is not a guarantee, but it is what we have been doing in the past with things like

field: SkipValidation[int] = None  # type: ignore

Using Field allows us to replace this with:

field: int = Field(default=None)

because pydantic does not validate the values of default by default. It's then up to us to ensure that the field has a value before initialisation is complete. The initialisation process looks like:

model_validator(mode='before')
field_validator(mode='before')
Inner validators. e.g. validation for types like int, str, ...
__post_init__.
field_validator(mode='after')
model_validator(mode='after')

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

zou3519 · 2025-10-21T14:21:37Z

vllm/config/vllm.py

+            if not (self.model_config is None or self.model_config.enforce_eager):
+                self.compilation_config.mode = CompilationMode.VLLM_COMPILE


The logic seems to have changed here. Did V0 deprecation proceed to a point that we can make these changes?

As far as I know, V0 is already gone

I think I've lived under a rock for too long, you're right - using the v0 engine just imports the v1 engine it looks like.

Some of the tests were relying on this behavior, but as long as they pass we're good

If any tests do rely on V0 defaults, I'll update them

mergify · 2025-10-23T00:20:04Z

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @hmellor.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

…or/vllm into mypy-attention-compilation

Fix mypy for vllm/attention and vllm/compilation

f823fa7

This change brings us closer to not needing a custom `mypy` check. Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

hmellor requested review from LucasWilkinson, ProExpertProg, WoosukKwon, houseroad, mgoin, robertgshaw2-redhat, simon-mo, tlrmchlsmth, yewentao256, youkaichao and zou3519 as code owners October 9, 2025 11:49

mergify bot added the v1 label Oct 9, 2025

Remove skipvalidation

ca03050

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

chatgpt-codex-connector bot reviewed Oct 9, 2025

View reviewed changes

vllm/attention/selector.py Outdated Show resolved Hide resolved

vllm/config/vllm.py Outdated Show resolved Hide resolved

gemini-code-assist bot reviewed Oct 9, 2025

View reviewed changes

vllm/config/vllm.py Outdated Show resolved Hide resolved

vllm/v1/attention/backends/utils.py Outdated Show resolved Hide resolved

Review comments

b6d07fa

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

hmellor requested review from NickLucche, bigPYJ1151 and jikunshang as code owners October 9, 2025 12:08

mergify bot added the tpu Related to Google TPUs label Oct 9, 2025

hmellor added 2 commits October 9, 2025 14:11

Review comments 2

5e0d99e

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Ignore modules that only exist after install

27f8ad8

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

hmellor commented Oct 9, 2025

View reviewed changes

vllm/config/device.py Outdated Show resolved Hide resolved

Remove unnecessary whitespace

1e59be7

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

hmellor commented Oct 9, 2025

View reviewed changes

vllm/config/compilation.py Outdated Show resolved Hide resolved

Remove now redundant cast

4574efc

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

mergify bot added the needs-rebase label Oct 9, 2025

ProExpertProg approved these changes Oct 9, 2025

View reviewed changes

vllm/config/compilation.py Outdated Show resolved Hide resolved

vllm/config/vllm.py Outdated Show resolved Hide resolved

vllm/config/compilation.py Outdated Show resolved Hide resolved

hmellor mentioned this pull request Oct 9, 2025

[Feature]: Fix all of the mypy check #26533

Open

5 tasks

Copilot AI mentioned this pull request Oct 9, 2025

[Feature] Move vllm/lora from SEPARATE_GROUPS to FILES in mypy config and fix type errors #26543

Closed

hmellor changed the title ~~Fix mypy for vllm/attention and vllm/compilation~~ [CI] Fix mypy for vllm/attention and vllm/compilation Oct 10, 2025

hmellor mentioned this pull request Oct 13, 2025

[FrontEnd] UNREVERT CompilationConfig overhaul (#20283): deprecate use_inductor in favor of backend, simplify custom_ops #26502

Merged

5 tasks

Merge branch 'main' into mypy-attention-compilation

9ec44ea

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

mergify bot removed the needs-rebase label Oct 16, 2025

Fix mypy errors

1170aa9

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

hmellor enabled auto-merge (squash) October 17, 2025 08:52

github-actions bot added the ready ONLY add when PR is ready to merge/full CI is needed label Oct 17, 2025

hmellor added 7 commits October 17, 2025 12:42

torch.Device -> torch.types.Device

80bb357

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Merge branch 'main' into mypy-attention-compilation

2d88ded

Review comment aboud compile_sizes

b456ee8

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Make it a before validator

60640a8

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Merge branch 'main' into mypy-attention-compilation

c249ed2

Skip None validation for delayed defaults

e19b02c

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Fix type hint for get_kv_cache_spec

65b3d74

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

zou3519 reviewed Oct 21, 2025

View reviewed changes

zou3519 approved these changes Oct 21, 2025

View reviewed changes

Merge branch 'main' into mypy-attention-compilation

a511f41

mergify bot added the needs-rebase label Oct 23, 2025

hmellor added 2 commits October 23, 2025 11:11

Merge branch 'main' into mypy-attention-compilation

de444cc

Signed-off-by: Harry Mellor <19981378+hmellor@users.noreply.github.com>

Merge branch 'mypy-attention-compilation' of https://github.com/hmell…

c76fc00

…or/vllm into mypy-attention-compilation

mergify bot removed the needs-rebase label Oct 23, 2025

		if not (self.model_config is None or self.model_config.enforce_eager):
		self.compilation_config.mode = CompilationMode.VLLM_COMPILE

Uh oh!

[CI] Fix mypy for vllm/attention and vllm/compilation #26482

Are you sure you want to change the base?

[CI] Fix mypy for vllm/attention and vllm/compilation #26482

Uh oh!

Conversation

hmellor commented Oct 9, 2025 • edited by github-actions bot Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

💡 Codex Review

Uh oh!

Uh oh!

Uh oh!

gemini-code-assist bot left a comment

Choose a reason for hiding this comment

Code Review

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hmellor Oct 9, 2025

Choose a reason for hiding this comment

Uh oh!

Uh oh!

mergify bot commented Oct 9, 2025

Uh oh!

ProExpertProg left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

Uh oh!

hmellor commented Oct 9, 2025

Uh oh!

hmellor commented Oct 9, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

zou3519 Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

hmellor Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

zou3519 Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

hmellor Oct 21, 2025

Choose a reason for hiding this comment

Uh oh!

mergify bot commented Oct 23, 2025

Uh oh!

Reviewers

Assignees

Labels

Projects

Milestone

Development

Uh oh!

4 participants

[CI] Fix mypy for `vllm/attention` and `vllm/compilation` #26482

[CI] Fix mypy for `vllm/attention` and `vllm/compilation` #26482

hmellor commented Oct 9, 2025 •

edited by github-actions bot

Loading

hmellor commented Oct 9, 2025 •

edited

Loading