SUPPORTED_MODELS.md

OpenVINO™ GenAI: Supported Models

Large language models

Architecture	Models	Example HuggingFace Models
ChatGLMModel	ChatGLM	THUDM/chatglm2-6b THUDM/chatglm3-6b
GemmaForCausalLM	Gemma	google/gemma-2b-it
GPTNeoXForCausalLM	Dolly	databricks/dolly-v2-3b
	RedPajama	ikala/redpajama-3b-chat
LlamaForCausalLM	Llama 3	meta-llama/Meta-Llama-3-8B meta-llama/Meta-Llama-3-8B-Instruct meta-llama/Meta-Llama-3-70B meta-llama/Meta-Llama-3-70B-Instruct
	Llama 2	meta-llama/Llama-2-13b-chat-hf meta-llama/Llama-2-13b-hf meta-llama/Llama-2-7b-chat-hf meta-llama/Llama-2-7b-hf meta-llama/Llama-2-70b-chat-hf meta-llama/Llama-2-70b-hf microsoft/Llama2-7b-WhoIsHarryPotter
	OpenLLaMA	openlm-research/open_llama_13b openlm-research/open_llama_3b openlm-research/open_llama_3b_v2 openlm-research/open_llama_7b openlm-research/open_llama_7b_v2
	TinyLlama	TinyLlama/TinyLlama-1.1B-Chat-v1.0
MistralForCausalLM	Mistral	mistralai/Mistral-7B-v0.1
	Notus	argilla/notus-7b-v1
	Zephyr	HuggingFaceH4/zephyr-7b-beta
PhiForCausalLM	Phi	microsoft/phi-2 microsoft/phi-1_5
QWenLMHeadModel	Qwen	Qwen/Qwen-7B-Chat Qwen/Qwen-7B-Chat-Int4 Qwen/Qwen1.5-7B-Chat Qwen/Qwen1.5-7B-Chat-GPTQ-Int4

The pipeline can work with other similar topologies produced by optimum-intel with the same model signature. The model is required to have the following inputs after the conversion:

1. input_ids contains the tokens.
2. attention_mask is filled with 1.
3. beam_idx selects beams.
4. position_ids (optional) encodes a position of currently generating token in the sequence and a single logits output.

Note

Models should belong to the same family and have the same tokenizers.

Text 2 image models

Architecture	Example HuggingFace Models
Latent Consistency Model	SimianLuo/LCM_Dreamshaper_v7
Stable Diffusion	botp/stable-diffusion-v1-5 dreamlike-art/dreamlike-anime-1.0 stabilityai/stable-diffusion-2 stabilityai/stable-diffusion-2-1
Stable Diffusion XL	stabilityai/stable-diffusion-xl-base-0.9 stabilityai/stable-diffusion-xl-base-1.0

Visual language models

Architecture	Models	Example HuggingFace Models
LLaVA	LLaVA-v1.5	llava-hf/llava-1.5-7b-hf
MiniCPMV	MiniCPM-V-2_6	openbmb/MiniCPM-V-2_6

Some models may require access request submission on the Hugging Face page to be downloaded.

If https://huggingface.co/ is down, the conversion step won't be able to download the models.