Add instructions for using Alpaca

[ggerganov]

See the work here: https://github.com/antimatter15/alpaca.cpp

There is no new functionality added, just a few hardcoded parameters in chat.cpp.
Instead of adding separate chat program, we should have an alpaca.py script that runs main with the respective parameters, so the user can simply run ./alpaca.py on the terminal.
It is a good time to start collecting prompts, so create a few useful Alpaca instruction prompts and place them in a prompts folder in the source tree. Make the alpaca.py script use one of them by default. Add option to change.

Add short instructions for using the alpaca.py for various tasks (translation, answering, .. whatever is popular) in the README and reference the alpaca.cpp repo for downloading the models.

[i-am-neo]

@ggerganov This would be a great addition - and maybe extended to wasm so the model can run on a browser if possible?

Big question though - the LLAMA weights themselves seem under restricted license, same with Alapaca.

[rabidcopy]

Just to chime in. I've had luck running the Alpaca model provided from that repository with this repository's code as is as a replacement for the 7B LlaMa model. Seems to work fair enough but I don't see it hurting to adding more info and functionality here.

[rabidcopy]

It appears the 13B Alpaca model provided from the alpaca.cpp repository cannot be loaded with llama.ccp however. I assume it expects the model to be in two parts.

llama_model_load: loading model from '../alpaca.cpp-master/ggml-alpaca-13b-q4.bin' - please wait ...
llama_model_load: n_vocab = 32000
llama_model_load: n_ctx   = 512
llama_model_load: n_embd  = 5120
llama_model_load: n_mult  = 256
llama_model_load: n_head  = 40
llama_model_load: n_layer = 40
llama_model_load: n_rot   = 128
llama_model_load: f16     = 2
llama_model_load: n_ff    = 13824
llama_model_load: n_parts = 2
llama_model_load: ggml ctx size = 8559.49 MB
llama_model_load: memory_size =   800.00 MB, n_mem = 20480
llama_model_load: loading model part 1/2 from '../alpaca.cpp-master/ggml-alpaca-13b-q4.bin'
llama_model_load: llama_model_load: tensor 'tok_embeddings.weight' has wrong size in model file
main: failed to load model from '../alpaca.cpp-master/ggml-alpaca-13b-q4.bin'

[Green-Sky]

It appears the 13B Alpaca model provided from the alpaca.cpp repository cannot be loaded with llama.ccp however. I assume it expects the model to be in two parts.

yes they are hardcoded right now. see thier patch antimatter15@97d327e

[ggerganov]

We can add a temporary cmd arg --alpaca-13b that if provided will override n_parts to 1. This way we will be compatible with all sorts of animals

[rabidcopy]

It appears the 13B Alpaca model provided from the alpaca.cpp repository cannot be loaded with llama.ccp however. I assume it expects the model to be in two parts.

yes they are hardcoded right now. see thier patch antimatter15@97d327e

Ah that does the trick, loaded the weights up fine with that change. A temporary argument for loading them also sounds cool.

[Green-Sky]

I think this model module count needs a more permanent solution. Either using some kind of auto-discover or a value written inside ggml file.

[i-am-neo]

We can add a temporary cmd arg --alpaca-13b that if provided will override n_parts to 1. This way we will be compatible with all sorts of animals

Vicunas here we come. 😉