Code Llama Fine-tuning Support

[adam-weinberger]

@JegernOUTT
This pull request adds support for fine-tuning Code Llama 7b. The biggest update is in the supported_models.py file which adds the configurations for Code Llama. The second biggest update adds support for additional model configurations (in finetune_train_default.py and finetune_train.py). The other updates are minor and are mostly for logging.

To test this update from command line:

1. Update the finetune.cfg file:

{
    "model_name": "codellama/7b",
    "use_heuristics": false,
    "train_steps": 75,
    "batch_size": 16,
    "save_every": 1
}

2. Put your code files (easiest is a .zip of the repo) in the folder: ~/.refact/perm-storage/uploaded-files
3. Run:

DEBUG=1 python -m refact.refact_data_pipeline.finetune.process_uploaded_files
DEBUG=1 python -m refact.refact_data_pipeline.finetune.finetune_filter
DEBUG=1 python -m refact.refact_data_pipeline.finetune.finetune_train

4. Verify the model has trained correctly by checking the "status" is "finished" in .refact/perm-storage/loras/lora-YYYYmmDD-MMSSss/status.json

To test this update from the UI:

1. Run:

DEBUG=1 python -m self_hosting_machinery.webgui.webgui
DEBUG=1 python -m self_hosting_machinery.inference.inference_worker --model codellama/7b

2. On the Model Hosting select "codellama/7b". Click through the "Sources" and "Finetune" tabs as normal, being sure To select the "codellama/7b" model under "Start New Fine Tune". NOTE: sometimes portions of the UI are grayed out when I try to go through this workflow and I'm not always able to completed. Is this expected? I have not touched any of the UI code.
3. Verify how the model has trained correctly by looking in the "Completed Runs"

[JegernOUTT]

@adam-weinberger please, rebase your branch onto https://github.com/smallcloudai/refact/tree/v1.2.0 and resolve all conflicts