Add corresponding lora finetuning with config scripts

[notoookay]

Hi, I don't know if it's necessary. I added the corresponding lora fine-tune scripts with config. These scripts work for the same purpose as scripts/finetune_with_accelerate_config.sh. The main changes include:

• Add lora and qlora fine-tune scripts with the config file.
• merge_lora is modified to be compatible with both scripts using config and old scripts, so the old scripts like scripts/finetune_lora_with_accelerate.sh still work.
• During merging, base_model will be loaded as bfloat16 if it was trained by lora. I found merged model dtype changed from bfloat16 to float32. The dtype is normal for qlora. I think you forgot to add torch_dtype=torch.bfloat16 for lora.
• Add config files (tulu2_7b_lora.yaml and tulu2_7b_qlora.yaml) as examples.

I have tested scripts using lora, qlora was not tested but should work fine.

[notoookay]

Please don't merge, there’s a bug during qlora merge. Working on it.

[notoookay]

Now it should work for both lora and qlora settings.

• Modify default pad_to_multiple_of to 8 in merge_lora.py, aligning with the setting in fine_tune.py.

[natolambert]

Thanks for helping with maitenance!