CacheConfig.block_size should always be int when used

[hmellor]

block_size should always be int by the time it is used. If it isn't errors would be thrown anyway.

This PR:

• removes Optional from the dataclass field so that type checkers don't complain that you can't do arithmetic with None
• improves the docstring explaining where this is normally set if the user doesn't set it

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[russellb]

lgtm

[markmc]

block_size should always be int by the time it is used. If it isn't errors would be thrown anyway.

This PR:

• removes Optional from the dataclass field so that type checkers don't complain that you can't do arithmetic with None

How about adding an assert block_size is not None instead? This way, rather than disabling the type checking, we make the type checker narrow the type after the assertion

[hmellor]

How about adding an assert block_size is not None instead? This way, rather than disabling the type checking, we make the type checker narrow the type after the assertion

To do that we'd have to put an assert in every place we read cache_config.block_size, which I don't think is very scalable/reliable. I tried putting one in __post_init__ but mypy does not see it either.

---

The reason I'm doing this is so we can do this:

# After fixing type errors resulting from follow_imports: "skip" -> "silent",
# move the directory here and remove it from tools/mypy.sh

for the v1 code.

By making this int instead of Optional[int] we remove ~20 errors that look similar to this one:

vllm/v1/attention/backends/flashinfer.py:468: error: Argument "page_size" to "FlashInferMetadata" has incompatible type "Optional[Literal[1, 8, 16, 32, 64, 128]]"; expected "int"  [arg-type]

---

Ideally, the defaults could be set in CacheConfig and we wouldn't have to defer them like this. But for now, ignoring one type here is the lesser of two evils IMO.