Merging AdaLora module with 8bit base model and 4bit basemodel

[LokeshJatangi]

Feature request

Hi @BenjaminBossan ,@younesbelkada, @pacman100

Support merging AdaLoRA adapters with base model when :

1. base model is loaded in int8 .
2. Base model loaded in 4bit using QLoRA method.

Motivation

• This is helpful when the goal is to merge adapter weights for faster inference with 8bit model and 4bit model inference
• Reduces memory usage when applying AdaLORA modules with LLMs like Falcon, LLAMA2 etc..

Your contribution

I am highly interested to add this feature to PEFT.
#875 and #851 PR's will be my reference for writing this PR.
Please suggests what unit tests needs to be added.
Also, provide your invaluable suggestions and pitfalls that I should avoid when implementing for AdaLoRA.

[BenjaminBossan]

Thanks for offering to take this task.

Please suggests what unit tests needs to be added.

I think you can take inspiration from the two PRs you quoted for determining what tests to add.

Also, provide your invaluable suggestions and pitfalls that I should avoid when implementing for AdaLoRA.

Hard to anticipate this. I would suggest to just go ahead and start with 8bit and see how far you get. Create a draft PR as soon as you got something going and we can give you feedback based on that.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.