-
Notifications
You must be signed in to change notification settings - Fork 536
[DOC] add LoRA user guide #1265
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Conversation
Signed-off-by: paulyu12 <507435917@qq.com>
Signed-off-by: paulyu12 <507435917@qq.com>
Signed-off-by: paulyu12 <507435917@qq.com>
|
This pull request has conflicts, please resolve those before we can evaluate the pull request. |
|
@paulyu12 Hi, thanks for the doc. I notice that most content is from vLLM doc. I think in vLLM Ascend doc, we don't need to re-write them again. For the most common part, we can add the link to vLLM directly. What we should write more is: what's the different usage on vLLM Ascend, for example, any model support list or limitation? Any feature + feature limitation? 2. How users use this feature with vLLM Ascend and so on. |
|
Actually, users can use LoRA with the user guide of vLLM doc. No extra limitation as I know. Because aclgraph mode doesn't support LoRA until now, maybe I'll add some tips about how to fallback to non-aclgraph mode. |
|
And the models that LoRA supports can be referred by vLLM Supported Models |
|
So the doc can be simple in vllm-ascend. Just add something like: Then add some limitation, like: does lora in vllm-ascend support graph mode? what model has been tested and so on. |
Signed-off-by: paulyu12 <507435917@qq.com>
### What this PR does / why we need it? Add LoRA user guide to DOC. The content refers to [LoRA Adapters](https://docs.vllm.ai/en/latest/features/lora.html). ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? No --------- Signed-off-by: paulyu12 <507435917@qq.com> Signed-off-by: ZhengWG <zwg0606@gmail.com>
### What this PR does / why we need it? Add LoRA user guide to DOC. The content refers to [LoRA Adapters](https://docs.vllm.ai/en/latest/features/lora.html). ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? No --------- Signed-off-by: paulyu12 <507435917@qq.com> Signed-off-by: ZhengWG <zwg0606@gmail.com>
### What this PR does / why we need it? Add LoRA user guide to DOC. The content refers to [LoRA Adapters](https://docs.vllm.ai/en/latest/features/lora.html). ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? No --------- Signed-off-by: paulyu12 <507435917@qq.com>
### What this PR does / why we need it? Add LoRA user guide to DOC. The content refers to [LoRA Adapters](https://docs.vllm.ai/en/latest/features/lora.html). ### Does this PR introduce _any_ user-facing change? No ### How was this patch tested? No --------- Signed-off-by: paulyu12 <507435917@qq.com>

What this PR does / why we need it?
Add LoRA user guide to DOC. The content refers to LoRA Adapters.
Does this PR introduce any user-facing change?
No
How was this patch tested?
No