add_weighted_adapter() is unusable, throws error: "Invalid type <class 'list'> found in target_modules"

[Vectorrent]

System Info

• transformers version: 4.34.0
• Platform: Linux-6.5.6-arch2-1-x86_64-with-glibc2.35
• Python version: 3.10.12
• Huggingface_hub version: 0.17.3
• Safetensors version: 0.4.0
• Accelerate version: 0.23.0
• Accelerate config: not found
• PyTorch version (GPU?): 2.1.0+cu121 (True)
• Tensorflow version (GPU?): not installed (NA)
• Flax version (CPU?/GPU?/TPU?): not installed (NA)
• Jax version: not installed
• JaxLib version: not installed
• Using GPU in script?: false
• Using distributed or parallel set-up in script?: false

Who can help?

@pacman100 @you

Information

• The official example scripts
• My own modified scripts

Tasks

• An officially supported task in the examples folder
• My own task or dataset (give details below)

Reproduction

Load the model:

model = AutoModelForCausalLM.from_pretrained(
        "RWKV/rwkv-4-169m-pile"
        device_map="auto"
    )

Convert it into a PeftModel (to make the add_weighted_adapter() method available):

model = PeftModel.from_pretrained(
        model, f"{adapter_dir}/{adapter}", adapter_name='adapter_1'
    )

Load the 2nd adapter:

model.load_adapter(
        f"{adapter_dir}/{adapter}", adapter_name='adapter_2'
    )

Attempt to merge the two adapters:

model.add_weighted_adapter(
        adapters=['adapter_1', 'adapter_2'],
        weights=[0.5, 0.5],
        adapter_name="combined",
        combination_type="svd",
    )

This will result in the error message:

Invalid type <class 'list'> found in target_modules

Expected behavior

I have 2 fully trained LoRA adapters. The only configuration difference between them is the rank and alpha used for each.

I would like to merge the two adapters into a new adapter, and set that adapter to the "active" one.

However, trying to use the add_weighted_adapter() method always results in the following error:

Invalid type <class 'list'> found in target_modules

I would expect the add_weighted_adapter() method to accept a list, for the "target_modules" argument.

If you can provide any advice, I would greatly appreciate it. I suspect that this is either unsupported and/or not fully-implemented; or, it has something to do with the way I'm attaching adapters. I've tried a bunch of alternate configurations, but I'm not having luck.

Thanks in advance for any help you might provide.

[BenjaminBossan]

Thanks for reporting. Could you please paste the full stacktrace?

[Vectorrent]

Thanks for the followup! Here's the stack trace:

vtx-lab-1   | Traceback (most recent call last):
vtx-lab-1   |   File "/src/lab/aigen/aigen/aigen.py", line 246, in __init__
vtx-lab-1   |     self.model.add_weighted_adapter(
vtx-lab-1   |   File "/usr/local/lib/python3.10/dist-packages/peft/tuners/lora/model.py", line 525, in add_weighted_adapter
vtx-lab-1   |     raise TypeError(f"Invalid type {target_module_types[0]} found in target_modules")
vtx-lab-1   | TypeError: Invalid type <class 'list'> found in target_modules

[BenjaminBossan]

I see, thanks. This will be fixed soon, once #1041 is merged.

[BenjaminBossan]

@LuciferianInk A fix has been merged. Could you please check if your issue is resolved when installing from latest PEFT main?

[Vectorrent]

Thanks! This issue does appear to be resolved.

That said, I have no idea how to use the merged adapter? If I go through all the steps from the original issue, and try these additional steps:

model.set_adapter("combined")
print(model.active_adapters)
  => ['adapter_1']

So, I'm still using adapter_1, and can't seem to switch to the newly-merged adapter. Am I missing something obvious, or is this another bug? If it is, I'd be happy to open an another issue.

EDIT: I think I figured it out. The model.active_adapters property will only return adapters loaded with model.add_adapter(). It is not updated when model.load_adapter() is used. Regardless, if I use model.delete_adapter() on all of my old adapters, after combining them into a new one (leaving just the combined adapter behind), the model appears to work correctly. Thus, it seems the combined adapter is being used.

[BenjaminBossan]

You are right, the output from model.active_adapters is not correct, the output from model.active_adapter (singular) should be correct though. Thanks for reporting. Regardless of that, the merge of the adapters should have worked and you should see the correct output.

Edit: Bugfix PR #1051.