[Lora] Cannot convert baichuan2 qlora to GGUF

[lizhiling12345]

Error: unrecognized sub-layer self_attn.W_pack in tensor base_model.model.model.layers.0.self_attn.W_pack.lora_A.weight

[slaren]

Can you test #3333?

[lizhiling12345]

Can you test #3333?

i use llama.cpp-lora-falcon,still error,
python convert-lora-to-ggml.py /adapter_model baichuan
Error: could not map tensor name base_model.model.model.layers.0.self_attn.W_pack.lora_A.weight
Note: the arch parameter must be specified if the model is not llama

[lizhiling12345]

Can you test #3333?

model is baichuan2,not baicun,can you teach me,how to fit baichuan2

[slaren]

Looks like supporting baichuan loras is going to be a bit tricky than that, because the attention weights need to be unpacked. Ie. the W_pack needs to be unpacked into attn_q, attn_k and attn_v, like convert-baichuan-hf-to-gguf.py does. I don't really know how that works, so I can't help here, sorry.

[lizhiling12345]

Looks like supporting baichuan loras is going to be a bit tricky than that, because the attention weights need to be unpacked. Ie. the W_pack needs to be unpacked into attn_q, attn_k and attn_v, like convert-baichuan-hf-to-gguf.py does. I don't really know how that works, so I can't help here, sorry.

Thanks for your reply

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.