Fine-tuning your own dataset

[mohamedae17]

I try to Fine-tuning a pre-trained models based on https://parl.ai/projects/recipes/
my data is just 5 lines :))

I run this line:
!parlai train_model -t fromfile:parlaiformat --fromfile_datapath /content/drive/MyDrive/sam.txt -m transformer/generator --multitask-weights 1,3,3,3 --init-model zoo:tutorial_transformer_generator/model --dict-file zoo:tutorial_transformer_generator/model.dict --embedding-size 512 --n-layers 8 --ffn-size 2048 --dropout 0.1 --n-heads 16 --learn-positional-embeddings True --n-positions 512 --variant xlm --activation gelu --skip-generation True --fp16 True --text-truncate 512 --label-truncate 128 --dict-tokenizer bpe --dict-lower True -lr 1e-06 --optimizer adamax --lr-scheduler reduceonplateau --gradient-clip 0.1 -veps 0.25 --betas 0.9,0.999 --update-freq 1 --attention-dropout 0.0 --relu-dropout 0.0 --skip-generation True -vp 15 -stim 60 -vme 20000 -bs 16 -vmt ppl -vmm min --save-after-valid True --model-file /content/drive/MyDrive/sammodel_finetuned -bs 4

and it take a long time and when it has ended it give me this error:

01:55:39 | new best ppl: 4.723 (previous best was 4.773)
01:55:39 | saving best valid model: /content/drive/MyDrive/sammodel_finetuned
Traceback (most recent call last):
  File "/usr/local/bin/parlai", line 8, in <module>
    sys.exit(main())
  File "/usr/local/lib/python3.10/dist-packages/parlai/__main__.py", line 14, in main
    superscript_main()
  File "/usr/local/lib/python3.10/dist-packages/parlai/core/script.py", line 325, in superscript_main
    return SCRIPT_REGISTRY[cmd].klass._run_from_parser_and_opt(opt, parser)
  File "/usr/local/lib/python3.10/dist-packages/parlai/core/script.py", line 108, in _run_from_parser_and_opt
    return script.run()
  File "/usr/local/lib/python3.10/dist-packages/parlai/scripts/train_model.py", line 1060, in run
    return self.train_loop.train()
  File "/usr/local/lib/python3.10/dist-packages/parlai/scripts/train_model.py", line 1010, in train
    for _train_log in self.train_steps():
  File "/usr/local/lib/python3.10/dist-packages/parlai/scripts/train_model.py", line 965, in train_steps
    stop_training = self.validate()
  File "/usr/local/lib/python3.10/dist-packages/parlai/scripts/train_model.py", line 598, in validate
    self.save_model()
  File "/usr/local/lib/python3.10/dist-packages/parlai/scripts/train_model.py", line 497, in save_model
    self.agent.save(fn)
  File "/usr/local/lib/python3.10/dist-packages/parlai/core/torch_agent.py", line 2034, in save
    atomic_save(states, path)
  File "/usr/local/lib/python3.10/dist-packages/parlai/utils/torch.py", line 59, in atomic_save
    os.replace(path + ".tmp", path)
FileNotFoundError: [Errno 2] No such file or directory: '/content/drive/MyDrive/sammodel_finetuned.tmp' -> '/content/drive/MyDrive/sammodel_finetuned'

Are there any advice for solve this error and make it faster ?

[klshuster]

Do you have write access to that directory?