[New Model]: when support MiniCPM3ForCausalLM MiniCPM3-4B model

[ML-GCN]

The model to consider.

https://huggingface.co/openbmb/MiniCPM3-4B

The closest model vllm already supports.

No response

What's your difficulty of supporting the model you want?

No response

Before submitting a new issue...

• Make sure you already searched for relevant issues, and asked the chatbot living at the bottom right corner of the documentation page, which can answer lots of frequently asked questions.

[DarkLight1337]

The model will be added to vLLM very soon, thanks @SUDA-HLT-ywfang!

[Mencuis]

When deploying MiniCPM3-4B using vllm, the inference speed is much slower than models of the same size, such as Qwen1.5-4B. Can you check this?

[DarkLight1337]

Can you run the HuggingFace implementation of those models and see if you observe the same differences?

[DarkLight1337]

It's difficult to narrow down the issue without providing more information.

[Mencuis]

Can you run the HuggingFace implementation of those models and see if you observe the same differences?

huggingface implementation on A800 without flash attention
new tokens 128
inference time

• MiniCPM3-4B 11.6 s
• Qwen1.5-4B 6.3 s

It seems that minicpm3 is indeed much slower