[Question]: slow with xInference

[Ty2000sdu]

Has this been raised before?

• I have checked the GitHub README.
• I have checked the Qwen documentation and cannot find an answer there.
• I have searched the issues and there is not a similar one.
• I confirm that this is not a bug report, a feature request, or a badcase.

Description

今天，用Xinference 在英伟达 4090 Gpu加载QWen2.5 14B，显存用了17G，好慢，不知道是不是Xinference没适配Qwen 2.5。

[jklj077]

Hi, Qwen2.5 is supported by Xinference.

1. Use Qwen2.5-14B-Instruct; don't use the base model for chat.
2. Did you use quantization? If so, say which one you used. By default, 14B models in bf16 should use at least 28GB VRAM. 4090 with 24GB is not enough.
3. Which backend of Xinference did you use? did you offload to CPU?