Guideline for generating correct lit_model.pth

[mkeya013]

Dear Author, is it possible to provide correct lit_model.pth file? Or a guideline to generate correct lit_model.pth file would be a great help.

Thank you.

[LinghaoChan]

@mkeya013 Thanks for your question.

1. git clone the `lit-gpt' and go into the corresponding dictionary.

git clone https://github.com/Lightning-AI/litgpt/tree/d78730a0694bff6c9f1ce285a6e8b471c1321cf5
cd ./litgpt/litgpt/scripts

To see all the available checkpoints for Vicuna, run:

python scripts/download.py | grep vicuna

which will print

lmsys/vicuna-7b-v1.3
lmsys/vicuna-13b-v1.3
lmsys/vicuna-33b-v1.3
lmsys/vicuna-7b-v1.5
lmsys/vicuna-7b-v1.5-16k
lmsys/vicuna-13b-v1.5
lmsys/vicuna-13b-v1.5-16k

2. Download the model.

In order to use a specific Vicuna checkpoint, for instance vicuna-7b-v1.5, download the weights and convert the checkpoint to the lit-gpt format:

pip install huggingface_hub

python scripts/download.py --repo_id lmsys/vicuna-7b-v1.5

python scripts/convert_hf_checkpoint.py --checkpoint_dir checkpoints/lmsys/vicuna-7b-v1.5

By default, the convert_hf_checkpoint step will use the data type of the HF checkpoint's parameters. In cases where RAM
or disk size is constrained, it might be useful to pass --dtype bfloat16 to convert all parameters into this smaller precision before continuing.

You're done! To execute the model just run:

pip install sentencepiece

python chat/base.py --checkpoint_dir checkpoints/lmsys/vicuna-7b-v1.5

---

This is the vicuna version I used. You can also follow the latest official guideline and revise it to vicuna-7b-1.5.

[mkeya013]

It worked. Thank you.