Adding a ReFT notebook to the tutorials section
#741
Conversation
| "\n", | ||
| "In this tutorial, we will be demonstrating how to fine-tune a language model using [Representation Finetuning for Language Models](https://arxiv.org/abs/2404.03592)\n", | ||
| "\n", | ||
| "We will use a traditional large language model and focus on fine tuning via ReFT adapters rather than the traditional full model fine tuning.\n", |
There was a problem hiding this comment.
i'm not sure "traditional large language model" is a good description here. maybe something like "lightweight encoder model" is a good description for Roberta nowadays.
| "source": [ | ||
| "### Model and Adapter initialization\n", | ||
| "\n", | ||
| "We load the `roberta-base` model along with the `LoReftConfig`. We can initalize a `reft` config with only one line of code, and can add it to our base model using the `add_adapter` function. On top of that, we can add a classification head to our adapter specifying 3 labels.\n", |
There was a problem hiding this comment.
could be nice to link our docs on ReFT here: https://docs.adapterhub.ml/methods.html#reft and mention that there is explanation on supported config parameters.
| " learning_rate=6e-4,\n", | ||
| " per_device_train_batch_size=32,\n", | ||
| " per_device_eval_batch_size=32,\n", | ||
| " num_train_epochs=2,\n", |
There was a problem hiding this comment.
please make a note that usually you'd train longer and this is only for demo purposes.
| " input_ids = tokenizer(text, truncation=True, padding='max_length')\n", | ||
| " input_ids[\"input_ids\"] = torch.tensor(input_ids[\"input_ids\"])\n", | ||
| " input_ids[\"attention_mask\"] = torch.tensor(input_ids[\"attention_mask\"])\n", |
There was a problem hiding this comment.
can add return_tensors="pt" (same as in preprocess_function) to tokenizer call so you don't need to convert to tensors afterwards
Sounds good, should the whisper notebook be added as well? I dont see it in the read.me |
Yes please, we forgot that 👍 |
This PR aims to add a tutorial notebook to utilize the
Loreftadapter to fine-tuneroberta-baseon themnlidataset.Reviews appreciated!