NameError: name 'PeftModel' is not defined

[labiadoa]

Getting this error when trying to fine tune "TheBloke/Llama-2-70B-Chat-GGML":

NameError Traceback (most recent call last)
in <cell line: 9>()
7
8 # get trainer
----> 9 trainer = SFTTrainer(
10 model_name,
11 train_dataset=dataset,

/usr/local/lib/python3.10/dist-packages/trl/trainer/sft_trainer.py in init(self, model, args, data_collator, train_dataset, eval_dataset, tokenizer, model_init, compute_metrics, callbacks, optimizers, preprocess_logits_for_metrics, peft_config, dataset_text_field, packing, formatting_func, max_seq_length, infinite, num_of_sequences, chars_per_token)
145 if callbacks is None:
146 callbacks = [PeftSavingCallback]
--> 147 elif not isinstance(model, (PreTrainedModel, PeftModel)):
148 model = AutoModelForCausalLM.from_pretrained(model)
149

NameError: name 'PeftModel' is not defined

I guess that is due to the type of model, but don't know exactly what. The point is that I want to fine tune a 70B version that fits into a 40 GB RAM GPU. The less the better (around 30 GB if possible), so quantization is inevitable I guess. It is possible to fine tune such model through TRL?

[younesbelkada]

Hi @labiadoa
I think you might have a broken installation of PEFT that led to that issue. You might want to create a new environment and re-install TRL and PEFT.
Also note that you cannot fine-tune a model that has been quantized with GGML, check out this section of the documentation: https://huggingface.co/docs/trl/main/en/sft_trainer#training-adapters to learn more about how to fine-tune adapters using PEFT on top of quantized models.

[github-actions]

This issue has been automatically marked as stale because it has not had recent activity. If you think this still needs to be addressed please comment on this thread.