[Bug]: msgspec.DecodeError: MessagePack data is malformed: trailing characters (byte 13)

[fangru-lin]

Your current environment

vllm==0.8.0, python=3.11

🐛 Describe the bug

I keep having this decode error when using vllm=0.8.0 for qwen-2.5 inference, I have the same error on both 1.5B and 7B model once I use apply-chat-template.

msgspec.DecodeError: MessagePack data is malformed: trailing characters (byte 13)

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[hlzhang109]

same

[lyx564]

same

[ViviqwerAsd]

same

[berserkr]

same:
159 Traceback (most recent call last):
160 File "/proj/checkpoints/bathen/envs/conda/olmes/lib/python3.10/threading.py", line 1016, in _bootstrap_inner
161 self.run()
162 File "/proj/checkpoints/bathen/envs/conda/olmes/lib/python3.10/threading.py", line 953, in run
163 self._target(*self._args, **self._kwargs)
164 File "/proj/checkpoints/bathen/envs/conda/olmes/lib/python3.10/site-packages/vllm/v1/engine/core_client.py", line 374, in process_outputs_socket
165 outputs = decoder.decode(frame.buffer)
166 File "/proj/checkpoints/bathen/envs/conda/olmes/lib/python3.10/site-packages/vllm/v1/serial_utils.py", line 34, in decode
167 return self.decoder.decode(obj)
168 msgspec.DecodeError: MessagePack data is malformed: trailing characters (byte 13)

[kxzxvbk]

same

[garified]

same

[soldni]

same.

[hans66hsu]

same

[pengyao96]

fp8 pertensor static quant in deepseekv2-lite, happend this bug

[kevinlu1248]

same

[ywang96]

FYI @njhill

[njhill]

Hi all, would it be possible to provide concrete steps to reproduce? And a full log containing the error if possible.

[kevinlu1248]

@njhill I can DM you the data you can use to repro it. Just shoot me an email at kevin [at] sweep.dev. I've found 3 prompts where you only get an error if you run them concurrently.

[njhill]

Would be useful if anyone else could share the request parameters used to trigger this, as well as precise vllm version and full log / tracebacks if possible, thanks!