vllm deploy error

[huazZeng]

Issue: ValueError in interns1_vit.py when serving intern-s1 model

Environment

• OS: Ubuntu 22.04
• CUDA: 12.8.1
• Python: 3.11
• PyTorch: 2.8.0
• vLLM: 0.11.0
• Transformers: 4.57.1

Command

vllm serve intern-s1 \
      --port 8008 \
      --host 0.0.0.0 \
      --served-model-name intern-s1 \
      --trust-remote-code \
      --gpu-memory-utilization 0.9 \
      --tensor-parallel-size 8

Error

File "/usr/local/lib/python3.11/site-packages/vllm/model_executor/models/interns1_vit.py", line 221, in forward
 (Worker_TP0 pid=1145) ERROR 10-24 16:40:22 [multiproc_executor.py:671]     B_, N_, H_, D_ = q.shape
(Worker_TP0 pid=1145) ERROR 10-24 16:40:22 [multiproc_executor.py:671]     ^^^^^^^^^^^^^^
(Worker_TP0 pid=1145) ERROR 10-24 16:40:22 [multiproc_executor.py:671] ValueError: not enough values to unpack (expected 4, got 3)

Description

When trying to serve the intern-s1 model using vLLM, the service fails with a ValueError in the interns1_vit.py file. The error occurs at line 221 where the code attempts to unpack 4 values from q.shape but only receives 3 values.

This suggests there's a shape mismatch in the tensor dimensions, likely related to the model's vision transformer implementation. The error occurs during the forward pass of the model execution.

Expected Behavior

The model should load and serve successfully without shape unpacking errors.

[lvhan028]

Fixed in vllm-project/vllm#23978

[AkaTsukijm]

i dont know why you guys closed this issue. the problem still exists.
File "/usr/local/lib/python3.10/dist-packages/vllm/model_executor/models/interns1_vit.py", line 221, in forward �[1;36m(Worker_TP7 pid=146)�[0;0m ERROR 11-06 02:51:33 [multiproc_executor.py:654] B_, N_, H_, D_ = q.shape �[1;36m(Worker_TP7 pid=146)�[0;0m ERROR 11-06 02:51:33 [multiproc_executor.py:654] ValueError: not enough values to unpack (expected 4, got 3) �[1;36m(Worker_TP6 pid=145)�[0;0m ERROR 11-06 02:51:33 [multiproc_executor.py:654] WorkerProc hit an exception.

[lvhan028]

Sorry, the mentioned PR above is wrong.
According to vLLM team, this issue is fixed by PR vllm-project/vllm#27480.
May try vLLM's latest main.
If it is still an issue, could you please leave a message under vllm-project/vllm#27480?

[AkaTsukijm]

Sorry, the mentioned PR above is wrong. According to vLLM team, this issue is fixed by PR vllm-project/vllm#27480. May try vLLM's latest main. If it is still an issue, could you please leave a message under vllm-project/vllm#27480?

Thanks for the reply. I searched for the cause of this error, and I found this issue before I left this comment. I definitely have already updated every packages that I need.

I am using Transformers 4.57.1 and vllm 0.11.0 and quite sure that they are all up-to-date.

[AkaTsukijm]

Maybe it will work in vLLM V0.11.1? (It has not been published though)

[lvhan028]

Yes. It was merged to vLLM main branch on Oct 25th which hasn't been released yet. You may follow vLLM's installation guide to build it from source or waiting for the next version.

Maybe it will work in vLLM V0.11.1? (It has not been published though)

[nilinykh]

I tried to use docker nightly builds to get access to the latest version that fixes this bug, but they seem to be baggy with the multi-node inference...I guess we need to wait for the 0.11.1 release.

[lvhan028]

I tried to use docker nightly builds to get access to the latest version that fixes this bug, but they seem to be baggy with the multi-node inference...I guess we need to wait for the 0.11.1 release.

Could you share this issue in PR vllm-project/vllm#27480?