Support starcoder family architectures (1B/3B/7B/13B) · Issue #3076 · ggerganov/llama.cpp

[irthomasthomas]

• Support starcoder family architectures (1B/3B/7B/13B) · Issue #3076 · ggerganov/llama.cpp

Previously, it wasn't recommended to incorporate non-llama architectures into llama.cpp. However, in light of the recent addition of the Falcon architecture (see Pull Request #2717), it might be worth reconsidering this stance.

One distinguishing feature of Starcoder is its ability to provide a complete series of models ranging from 1B to 13B. This capability can prove highly beneficial for speculative decoding and making coding models available for edge devices (e.g., M1/M2 Macs).

I can contribute the PR if it matches llama.cpp's roadmap.

Suggested labels

{ "key": "LLM-Applications", "value": "Practical applications of Large Language Models, such as edge device coding models and speculative decoding" } { "key": "Multimodal-LM", "value": "LLMs that combine modes such as text and image recognition" }