Are some layers frozen while fine-tuning？

[TccccD]

Hello, when finetuning LLaMa, are there any specific layers frozen so that these layers do not trained in training?
@percyliang @guestrin @thashim @rtaori @lxuechen

[lxuechen]

Hi, thanks for your interest in our work!

The current batch of models are fine-tuned by optimizing all parameters. We're still in the process of testing light-weight fine-tuned models.

[iseesaw]

Hi, thanks for your interest in our work!

The current batch of models are fine-tuned by optimizing all parameters. We're still in the process of testing light-weight fine-tuned models.

Hi, have you tried parameter-efficient fine-tuning methods?

[lxuechen]

Hi, thanks for your interest in our work!
The current batch of models are fine-tuned by optimizing all parameters. We're still in the process of testing light-weight fine-tuned models.

Hi, have you tried parameter-efficient fine-tuning methods?

Yes. That's what I meant by saying light-weight fine-tuning methods.