question about Pretrained LLAMA applicable to Llama_adapter model. thanks

[jzssz]

1. I noticed that the pretrained llama model in the code is in .pth format. So if I have a pretrained llama model in .bin format, can I use it as a Llama base model to train llama-adapter v2 (put it in "--llama_path") ?

2. If I have a llama (7B) pre-trained with LORA (get a light-weight model like"adapter_model.bin"), can I use it as a Llama base model to train llama-adapter v2 (put it in "--llama_path") ?

thanks a lot

[csuhan]

I guess your checkpoint is transformers-based, while our code only support the original llama format (https://github.com/facebookresearch/llama). So you need to download the original llama weights instead of transformers format.