[Bug]: _Z35cutlass_blockwise_scaled_grouped_mmRN2at6TensorERKS0_S3_S3_S3_S3_S3_

[fernandaspets]

Your current environment

2 x RTX 6000 PRO

python -m vllm.entrypoints.api_server --model cognitivecomputations/Qwen3-235B-A22B-AWQ --enable-reasoning --reasoning-parser deepseek_r1 -tp 2

ImportError: /home/giga/vllm/vllm/_C.abi3.so: undefined symbol: _Z35cutlass_blockwise_scaled_grouped_mmRN2at6TensorERKS0_S3_S3_S3_S3_S3_

How you are installing vllm

Verified NVIDIA-SMI for Driver 575.62 and CUDA 12.9

git clone https://github.com/vllm-project/vllm.git
cd vllm
python -m venv vllm
source ./vllm/bin/activate

pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu129  # note changed cu128 to cu129

python use_existing_torch.py
python -m pip install -r requirements/build.txt
python -m pip install -r requirements/common.txt
python -m pip install -e . --no-build-isolation -v

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[rashmibanthia]

I'm also struggling with similar error - Here are the environment details -

(1) Error message

ImportError: vllm/vllm/_C.abi3.so: undefined symbol: _Z35cutlass_blockwise_scaled_grouped_mmRN2at6TensorERKS0_S3_S3_S3_S3_S3_

(2) Nvidia 5090 - Driver Version: 575.57.08 CUDA Version: 12.9

(3) torch Version: 2.9.0.dev20250704+cu128

(4) vLLM build from source - Version: 0.9.2rc2.dev25+g32c9be220.d20250705.cu129

git clone https://github.com/vllm-project/vllm.git
cd vllm
python use_existing_torch.py
pip install -r requirements/build.txt
export MAX_JOBS=6
pip install --no-build-isolation -e .

[rashmibanthia]

I'm also struggling with similar error - Here are the environment details -

(1) Error message

ImportError: vllm/vllm/_C.abi3.so: undefined symbol: _Z35cutlass_blockwise_scaled_grouped_mmRN2at6TensorERKS0_S3_S3_S3_S3_S3_

(2) Nvidia 5090 - Driver Version: 575.57.08 CUDA Version: 12.9

(3) torch Version: 2.9.0.dev20250704+cu128

(4) vLLM build from source - Version: 0.9.2rc2.dev25+g32c9be220.d20250705.cu129

git clone https://github.com/vllm-project/vllm.git
cd vllm
python use_existing_torch.py
pip install -r requirements/build.txt
export MAX_JOBS=6
pip install --no-build-isolation -e .

---

This is resolved - I have following versions now

(2) Nvidia 5090 - Driver Version: 575.57.08 CUDA Version: 12.9 (Same as before)

(3) torch Version: 2.9.0.dev20250705+cu128

(4) vLLM build from source - Version: 0.9.2rc2.dev35+g43813e636.d20250706.cu129

[fernandaspets]

wow cool how did you resolve it??

[rashmibanthia]

wow cool how did you resolve it??

Just installed PyTorch nightly pip3 install --pre torch torchvision torchaudio --index-url https://download.pytorch.org/whl/nightly/cu128 in a new environment.

Also build from source vLLM (They must have updated something cutlass related and I'm guessing it resolved it - can't wait for stable release)

[fernandaspets]

That's great I noticed there is a PyTorch nightly cu129. they don't show it on the website but I found it by switching the 8 for 9. That's the one I used yesterday. But so I did that because I was getting an error that PyTorch was expecting 12.8 when I used the one you are using (cu128 nightly) did you get that compile error and if so how did you solve it?

[LucasWilkinson]

Can you test with: #20513

[fernandaspets]

Can you test with: #20513

Seems resolved today in the main branch. I successfully built it today in a fresh pyenv environment with Python 3.12.11. And it seems to get past that initial error. It's downloading a model and I am trying with -tp 2 so lets see if it works. The same model worked yesterday using NVIDIAs image 26.06 with the preinstalled PyTorch 2.8 and vLLM 0.90. So let's see if it works here with bleeding edge PyTorch 2.9, vLLM 0.92 and CUDA 12.9.1