Zaks repository of openllm

This repo (on main branch) is already included by openllm by default.

If you want more up-to-date untested models, please add our nightly branch.

openllm repo add nightly https://github.com/bentoml/openllm@nightly

Supported Models

Table of Contents

• Llama-3
• Phi-3
• Mistral
• Qwen-2
• Gemma
• Llama-2
• Mixtral

---

Llama-3

Model	Version	Huggingface Link
llama3	70b-instruct-awq-4bit-e968	HF Link
llama3	70b-instruct-fp16-6aed	HF Link
llama3	8b-instruct-awq-4bit-f9de	HF Link
llama3	8b-instruct-fp16-f703	HF Link

---

Phi-3

Model	Version	Huggingface Link
phi3	3.8b-instruct-fp16-30b8	HF Link
phi3	3.8b-instruct-ggml-q4-f5db	HF Link

---

Mistral

Model	Version	Huggingface Link
mistral	7b-instruct-awq-4bit-0850	HF Link
mistral	7b-instruct-fp16-ac2b	HF Link

---

Qwen-2

Model	Version	Huggingface Link
qwen2	0.5b-instruct-fp16-fcc6	HF Link
qwen2	1.5b-instruct-fp16-50d8	HF Link
qwen2	57b-a14b-instruct-fp16-3f06	HF Link
qwen2	72b-instruct-awq-4bit-15fd	HF Link
qwen2	72b-instruct-fp16-7b44	HF Link
qwen2	7b-instruct-awq-4bit-ce1b	HF Link
qwen2	7b-instruct-fp16-844c	HF Link

---

Gemma

Model	Version	Huggingface Link
gemma	2b-instruct-fp16-0856	HF Link
gemma	7b-instruct-awq-4bit-d11b	HF Link
gemma	7b-instruct-fp16-3e1c	HF Link

---

Llama-2

Model	Version	Huggingface Link
llama2	13b-chat-fp16-921b	HF Link
llama2	70b-chat-fp16-258c	HF Link
llama2	7b-chat-awq-4bit-8df2	HF Link
llama2	7b-chat-fp16-2e3a	HF Link

---

Mixtral

Model	Version	Huggingface Link
mixtral	8x7b-instruct-v0.1-awq-4bit-2953	HF Link
mixtral	8x7b-instruct-v0.1-fp16-71c6	HF Link

---