Problem with merging AdaLoRA adapters

[h4iku]

System Info

peft @ git+https://github.com/huggingface/peft@c8974c5880b28a913e35f050e82402e34d181c63
accelerate==0.29.2
transformers==4.39.3

Ubuntu 22.04.4 LTS
Python 3.11.9

Who can help?

@BenjaminBossan @pacman100

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder
• My own task or dataset (give details below)

Reproduction

I am trying to merge two AdaLoRA checkpoints of the same model and have written this script for it:

from peft import PeftConfig, PeftModel
from transformers import AutoModelForSeq2SeqLM

model_id = "Salesforce/codet5-small"
config = PeftConfig.from_pretrained("checkpoint-6621")
model = AutoModelForSeq2SeqLM.from_pretrained(config.base_model_name_or_path, device_map="auto")

model = PeftModel.from_pretrained(model, "checkpoint-6621", adapter_name="checkpoint-6621")
model.load_adapter("checkpoint-13242", adapter_name="checkpoint-13242")

adapters = ["checkpoint-6621", "checkpoint-13242"]
weights = [1.0, 1.0]
adapter_name = "merged"
density = 0.2
model.add_weighted_adapter(adapters, weights, adapter_name, combination_type="ties", density=density)

model.set_adapter("merged")
print(model.active_adapters)

When I run the script, I get this error:

Traceback (most recent call last):
  File "merge.py", line 15, in <module>
    model.add_weighted_adapter(adapters, weights, adapter_name, combination_type="ties", density=density)
  File ".venv/lib/python3.11/site-packages/peft/tuners/lora/model.py", line 579, in add_weighted_adapter
    combination_type, new_rank, new_target_modules = self._check_add_weighted_adapter(
                                                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File ".venv/lib/python3.11/site-packages/peft/tuners/lora/model.py", line 516, in _check_add_weighted_adapter
    raise TypeError(f"Invalid type {target_module_types[0]} found in target_modules")
TypeError: Invalid type <class 'list'> found in target_modules

The given error is similar to this issue: #1045

Expected behavior

It is expected that the adapters merge successfully. The same script works for LoRA adapters, resulting in a merged adapter.

[BenjaminBossan]

The error you encounter is indeed a bug, which can be fixed by adding the lines:

model.peft_config["checkpoint-6621"].target_modules = set(model.peft_config["checkpoint-6621"].target_modules)
model.peft_config["checkpoint-13242"].target_modules = set(model.peft_config["checkpoint-13242"].target_modules)

after model.load_adapter("checkpoint-13242", adapter_name="checkpoint-13242").

However, even with this workaround, this won't work because add_weighted_adapter is only supported for LoRA, not AdaLoRA.