Skip to content

Latest commit

 

History

History
267 lines (185 loc) · 6.08 KB

api_ref_models.rst

File metadata and controls

267 lines (185 loc) · 6.08 KB

torchtune.models

.. currentmodule:: torchtune.models

llama3 & llama3.1

All models from the Llama3 family.

Request Access on Hugging Face.

To download the Llama3-8B-Instruct model:

tune download meta-llama/Meta-Llama-3-8B-Instruct --hf-token <HF_TOKEN>

To download the Llama3-70B-Instruct model:

tune download meta-llama/Meta-Llama-3-70B-Instruct --ignore-patterns "original/consolidated*" --hf-token <HF_TOKEN>

To download the Llama3.1 weights of the above models, you can instead download from Meta-Llama-3.1-8B-Instruct, Meta-Llama-3.1-70B-Instruct, or Meta-Llama-3.1-405B-Instruct.

.. autosummary::
    :toctree: generated/
    :nosignatures:

    llama3.llama3
    llama3.lora_llama3
    llama3.llama3_8b
    llama3.lora_llama3_8b
    llama3.qlora_llama3_8b
    llama3.llama3_70b
    llama3.lora_llama3_70b
    llama3.qlora_llama3_70b
    llama3.llama3_tokenizer
    llama3.Llama3Tokenizer

    |

    llama3_1.llama3_1
    llama3_1.lora_llama3_1
    llama3_1.llama3_1_8b
    llama3_1.lora_llama3_1_8b
    llama3_1.qlora_llama3_1_8b
    llama3_1.llama3_1_70b
    llama3_1.lora_llama3_1_70b
    llama3_1.qlora_llama3_1_70b
    llama3_1.llama3_1_405b

Note

The Llama3.1 tokenizer reuses the llama3.llama3_tokenizer builder class.

llama2

All models from the Llama2 family.

Request Access on Hugging Face.

To download the Llama2-7B model:

tune download meta-llama/Llama-2-7b-hf --hf-token <HF_TOKEN>

To download the Llama2-13B model:

tune download meta-llama/Llama-2-13b-hf --hf-token <HF_TOKEN>

To download the Llama2-70B model:

tune download meta-llama/Llama-2-70b-hf --hf-token <HF_TOKEN>
.. autosummary::
    :toctree: generated/
    :nosignatures:

    llama2.llama2
    llama2.lora_llama2
    llama2.llama2_7b
    llama2.lora_llama2_7b
    llama2.qlora_llama2_7b
    llama2.llama2_13b
    llama2.lora_llama2_13b
    llama2.qlora_llama2_13b
    llama2.llama2_70b
    llama2.lora_llama2_70b
    llama2.qlora_llama2_70b
    llama2.llama2_tokenizer
    llama2.Llama2Tokenizer
    llama2.llama2_reward_7b
    llama2.lora_llama2_reward_7b
    llama2.qlora_llama2_reward_7b
    llama2.Llama2ChatTemplate

code llama

Models from the Code Llama family.

Request Access on Hugging Face.

To download the CodeLlama-7B model:

tune download codellama/CodeLlama-7b-hf --hf-token <HF_TOKEN>
.. autosummary::
    :toctree: generated/
    :nosignatures:

    code_llama2.code_llama2_7b
    code_llama2.lora_code_llama2_7b
    code_llama2.qlora_code_llama2_7b
    code_llama2.code_llama2_13b
    code_llama2.lora_code_llama2_13b
    code_llama2.qlora_code_llama2_13b
    code_llama2.code_llama2_70b
    code_llama2.lora_code_llama2_70b
    code_llama2.qlora_code_llama2_70b

qwen-2

Models of size 0.5B, 1.5B, and 7B from the Qwen2 family.

To download the Qwen2 1.5B model, for example:

tune download Qwen/Qwen2-1.5B-Instruct --output-dir /tmp/Qwen2-1.5B-Instruct --ignore-patterns None
.. autosummary::
    :toctree: generated/
    :nosignatures:

    qwen2.qwen2
    qwen2.lora_qwen2
    qwen2.qwen2_7b
    qwen2.qwen2_0_5b
    qwen2.qwen2_1_5b
    qwen2.lora_qwen2_7b
    qwen2.lora_qwen2_0_5b
    qwen2.lora_qwen2_1_5b
    qwen2.qwen2_tokenizer
    qwen2.Qwen2Tokenizer

phi-3

Models from the Phi-3 mini family.

To download the Phi-3 Mini 4k instruct model:

tune download microsoft/Phi-3-mini-4k-instruct --ignore-patterns None --hf-token <HF_TOKEN>
.. autosummary::
    :toctree: generated/
    :nosignatures:

    phi3.phi3
    phi3.lora_phi3
    phi3.phi3_mini
    phi3.lora_phi3_mini
    phi3.qlora_phi3_mini
    phi3.phi3_mini_tokenizer
    phi3.Phi3MiniTokenizer

mistral

All models from Mistral AI family.

Request Access on Hugging Face.

To download the Mistral 7B v0.1 model:

tune download mistralai/Mistral-7B-v0.1 --hf-token <HF_TOKEN>
.. autosummary::
    :toctree: generated/
    :nosignatures:

    mistral.mistral
    mistral.lora_mistral
    mistral.mistral_classifier
    mistral.lora_mistral_classifier
    mistral.mistral_7b
    mistral.lora_mistral_7b
    mistral.qlora_mistral_7b
    mistral.mistral_reward_7b
    mistral.lora_mistral_reward_7b
    mistral.qlora_mistral_reward_7b
    mistral.mistral_tokenizer
    mistral.MistralTokenizer
    mistral.MistralChatTemplate

gemma

Models of size 2B and 7B from the Gemma family.

Request Access on Hugging Face.

To download the Gemma 2B model:

tune download google/gemma-2b --ignore-patterns None  --hf-token <HF_TOKEN>

To download the Gemma 7B model:

tune download google/gemma-7b --ignore-patterns "gemma-7b.gguf"  --hf-token <HF_TOKEN>
.. autosummary::
    :toctree: generated/
    :nosignatures:

    gemma.gemma
    gemma.lora_gemma
    gemma.gemma_2b
    gemma.lora_gemma_2b
    gemma.qlora_gemma_2b
    gemma.gemma_7b
    gemma.lora_gemma_7b
    gemma.qlora_gemma_7b
    gemma.gemma_tokenizer
    gemma.GemmaTokenizer

clip

Vision components to support multimodality using CLIP encoder.

.. autosummary::
    :toctree: generated/
    :nosignatures:

    clip.clip_vision_encoder
    clip.TokenPositionalEmbedding
    clip.TiledTokenPositionalEmbedding
    clip.TilePositionalEmbedding