difference between get_peft_model and from_pretrained

[nuoma]

Hi, when I'm training LLAMA with Lora, i seem to get different results when loading Lora weights via get_peft_model and from_pretrained. Can't really tell why,both can infer successfully, but with vastly different results.

Could someone be so kind to tell me which is the correct way of doing this. Many thanks!

[yyqi17]

Hi nuoma, could you please show me your code in saving & loading checkpoints? I'm finetuning llama with lora, and seem to encounter some issues when infering my checkpoint (saved by model.save_pretrained() and loaded by PeftModel.from_pretrained(base_mode, lora_ckpt_path)). Thanks a lot!

[ingo-m]

Possibly related? #503 🤔

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.