如何部署Qwen的预训练模型，加速推理
#1123
Replies: 2 comments
-
|
尝试使用vllm加载预训练模型，模型回答很乱，有没有推荐的启动参数 |
Beta Was this translation helpful? Give feedback.
0 replies
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
-
通过原生的transformers推理非常慢，有没有什么方式可以加速推理？
Beta Was this translation helpful? Give feedback.
All reactions