How to add an adapter to a quantized model without peft?

[mkgs210]

Environment info

• adapters version: 0.1.1
• Platform: Windows 10
• Python version: 3.11.0
• PyTorch version (GPU?): 2.1.0
• Using GPU in script?: Yes
• Using distributed or parallel set-up in script?: No

Information

Model I am using "ai-forever/rugpt3large_based_on_gpt2":

Language I am using the model on Russian:

Adapter setup I am using (if any): BnConfig, SeqBnConfig, DoubleSeqBnConfig, PrefixTuningConfig, LoRAConfig, IA3Config, PromptTuningConfig, MAMConfig, UniPELTConfig

The problem arises when using:

• the official example scripts:
• my own modified scripts: I used https://github.com/adapter-hub/adapters/blob/main/notebooks/Text_Generation_Training.ipynb
  and https://colab.research.google.com/drive/1VoYNfYDKcKRQRor98Zbf2-9VQTtGJ24k?usp=sharing&pli=1#scrollTo=Ybeyl20n3dYH

The tasks I am working on is:

• an official GLUE/SQUaD task: text generation
• my own task or dataset:

To reproduce

Steps to reproduce the behavior:

1. Use the Text_Generation_Training adapters example
2. Load model in 4bit or 8bit
3. delete "init(model)" because it is not working and replace AutoModelForCausalLM by AutoAdapterModel
4. trainer.train

Expected behavior

Fine-tuning the model

Real behavior

I'm trying to add an adapter to a quantized model. I would like to use not only the LORA adapter available in PEFT, but also other adapters.
However, as soon as I run the training, the error appears:

ValueError: You cannot perform fine-tuning on purely quantized models. Please attach trainable adapters on top of the quantized model to correctly perform fine-tuning. Please see: https://huggingface.co/docs/transformers/peft for more details

i tried to use PEFT features like prepare_model_for_kbit_training but couldn't add non PEFT adapter

[calpt]

Hey @mkgs210, training on quantized models in the style of e.g. QLoRA is not currently supported by the released version ofadapters. There's a WIP pull request for adding this support here though: #663.

This PR also adds a colab notebook to demo finetuning Llama 2 with QLoRA and adapters: https://github.com/calpt/adapter-transformers/blob/dev/qlora/notebooks/QLoRA_Llama2_Finetuning.ipynb.

Would love for you to test this out and help us adding support for training quantized models to adapters, thanks!