-
Notifications
You must be signed in to change notification settings - Fork 1.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
Support Multi-LoRA/qLoRA in PEFT #1005
Comments
Thanks for bringing this to our attention. I have a couple of questions and comments:
|
We have experiment results for memory usage at README.md, shows the peak memory usage of the existing method(Alpaca-LoRA) compared to our method on one NVIDIA RTX A6000 GPU. Alpaca-LoRA triggered an OOM error after 4 parallel tasks, while our method can handle twice that amount. |
Also, in terms of time cost, we achieved approximately 5% faster results than the sequential execution of Alpaca-LoRA, which utilizes peft as its implementation. |
@BenjaminBossan, "Automatic Parameter Learning" and other enhancements, such as early stopping, are extended features built upon our core library, offering a more convenient fine-tuning solution. However, these functions are not yet fully completed; the performance improvements I mentioned earlier remain our key selling points. |
Ah, thanks for that. Is the script to create this benchmark contained in the repo somewhere? |
This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread. |
Sorry forgot to response. We have put our experimental code at https://github.com/yezhem/aspen-experiment |
Feature request
Dear All
We are woking on improving the GPU memory usage for multi-lora fine tune. As you know, different LLM models with same base model can share the GPU memory in theory. based on this motivation, we had done some experiment results and show that it indeed improve the memory greatly. our code repo is here: https://github.com/TUDB-Labs/multi-lora-fine-tune
any comments are welcome.
Motivation
save the GPU memory usage
Your contribution
can we send our improvement into this project or how we can implement this under this framework? any suggestions are welcome.
The text was updated successfully, but these errors were encountered: