[CI/Build] Update the Dockerfile to include Blackwell archs

[0xjunhao]

Update the Dockerfile to include Blackwell archs and use ubuntu 24.04 as base image

Have tested that this works for RTX5090.
Test build FYI: ubicloud/vllm-openai:latest

[github-actions]

👋 Hi! Thank you for contributing to the vLLM project.

💬 Join our developer Slack at https://slack.vllm.ai to discuss your PR in #pr-reviews, coordinate on features in #feat- channels, or join special interest groups in #sig- channels.

Just a reminder: PRs would not trigger full CI run by default. Instead, it would only run fastcheck CI which starts running only a small and essential subset of CI tests to quickly catch errors. You can run other CI tests on top of those by going to your fastcheck build on Buildkite UI (linked in the PR checks section) and unblock them. If you do not have permission to unblock, ping simon-mo or khluu to add you in our Buildkite org.

Once the PR is approved and ready to go, your PR reviewer(s) can run CI to test the changes comprehensively before merging.

To run CI, PR reviewers can either: Add ready label to the PR or enable auto-merge.

🚀

[simon-mo]

LGTM! Let's see if it builds, and i'll poll contributor on the ubuntu 24 change

Seems like Ubuntu24 upgrade might break people's workflow

[simon-mo]

Is it possible to do this without updating base image?

[tlrmchlsmth]

We need to stick to building with older OSes, unfortunately. The reason for this is that the glibc version is forwards-compatible but not backwards-compatible. If we upgrade to Ubuntu 24.04, then vLLM won't work on 22.04, for instance.

Can the CUDA arch list be extended without upgrading Ubuntu?

[0xjunhao]

I think so. Cuda 12.8 still supports 20.04 fortunately. There are two versions there, the base was on 20.04, and vllm-base was on 22.04. Should I revert both?

[0xjunhao]

FYI ubuntu 20.04 will reach its EOL on May 31, 2025, which is only two weeks away.

[tlrmchlsmth]

FYI ubuntu 20.04 will reach its EOL on May 31, 2025, which is only two weeks away.

I think we should consider the OS upgrade separately. Upgrading to Ubuntu 22.04 will put us on glibc 2.35.

This will break vLLM on, for instance:

• Debian bullseye, which is on 2.31 and not EoL until Aug 2026
• UBI 9, which is on 2.34

[0xjunhao]

It seems that with the new archs, the image build check is timing out during the FlashInfer stage. Is there a way to increase the timeout limit?

[alew3]

@0xjunhao I tried running the model OpenGVLab/InternVL3-1B-Instruct on an RTX 5090 with the dockerfile ubicloud/vllm-openai:latest and got this error

CUDA error (/__w/xformers/xformers/third_party/flash-attention/hopper/flash_fwd_launch_template.h:175): no kernel image is available for execution on the device

[cchadowitz]

@0xjunhao I tried running the model OpenGVLab/InternVL3-1B-Instruct on an RTX 5090 with the dockerfile ubicloud/vllm-openai:latest and got this error

CUDA error (/__w/xformers/xformers/third_party/flash-attention/hopper/flash_fwd_launch_template.h:175): no kernel image is available for execution on the device

I believe I could work around this by building the latest xformers from source.

[0xjunhao]

Discussed with Simon, closing this PR. Please refer to PR #18095.