[UX] Add FlashInfer as default CUDA dependency

[mgoin]

Purpose

It seems that FlashInfer does not require nvcc to installed from source anymore since flashinfer-python>=0.2.9, so we can move it to be a default dependency!

Obviously to have it JIT compile kernels it needs to have nvcc available, so we will currently add that condition to has_flashinfer(). This is more conservative than it needs to be, as if we have an AOT compiled wheel installed there is likely no need for nvcc, but we can improve this later.

Also with flashinfer-python==0.4.0, we can now move to using flashinfer-cubin and flashinfer-jit-cache wheels instead of pre-compiling ourselves in the docker!

Test Plan

Test Result

I used a uv python image to simulate installing flashinfer-python without any cuda toolkit available

docker run -it --rm astral/uv:bookworm-slim bash
root@bbbb3b887d99:/# uv venv --python 3.12
root@bbbb3b887d99:/# source .venv/bin/activate
(.venv) root@bbbb3b887d99:/# uv pip install flashinfer-python==0.2.8
...
      FileNotFoundError: [Errno 2] No such file or directory: 'nvcc'

(.venv) root@bbbb3b887d99:/# uv pip install flashinfer-python==0.2.9
Installed 41 packages in 451ms

(.venv) root@bbbb3b887d99:/# uv pip install flashinfer-python==0.3.1
Installed 5 packages in 103ms

---

Essential Elements of an Effective PR Description Checklist

• The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
• The test plan, such as providing test command.
• The test results, such as pasting the results comparison before and after, or e2e results
• (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
• (Optional) Release notes update. If your change is user facing, please update the release notes draft in the Google Doc.

[gemini-code-assist]

Code Review

This pull request aims to make FlashInfer a default CUDA dependency, which is a great step for user experience. My main concern is with the change in vllm/utils/flashinfer.py, which could negatively impact users in environments without the full CUDA toolkit. Please see my detailed comment.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting

• @codex fix this CI failure
• @codex address that feedback

[bbartels]

Do we still need the flashinfer compilation at all? Flashinfer as of today releases a nightly wheel with full precompiled cubins.

[mgoin]

@bbartels We will start using the precompiled wheels (flashinfer-jit-cache and flashinfer-cubins) in the upcoming flashinfer 0.4.0 release. However I doubt we will include them by default since they don't live on pypi and are ~1 GB, which is a sizable increase for all users

[bbartels]

That's fair, might not be a good idea to require them for the vllm wheel. But I could see it be worth including it by default in the Dockerfile, since that would unblock moving back to the substantially smaller nvidia base image.
(Unless I am mistaken)

[mergify]

This pull request has merge conflicts that must be resolved before it can be
merged. Please rebase the PR, @mgoin.

https://docs.github.com/en/pull-requests/collaborating-with-pull-requests/working-with-forks/syncing-a-fork

[yewentao256]

LGTM, thanks for the work!