Skip to content

Commit

Permalink
remove unnecessary files
Browse files Browse the repository at this point in the history
  • Loading branch information
hakotesova committed Dec 9, 2024
1 parent 0c0b38f commit 2a949a4
Show file tree
Hide file tree
Showing 6 changed files with 75 additions and 20 deletions.
15 changes: 15 additions & 0 deletions sdk/python/foundation-models/system/distillation/README.md
Original file line number Diff line number Diff line change
Expand Up @@ -14,6 +14,7 @@ description: An explanation on model distillaton and step-by-step guide on creat
- [Welcome](#welcome)
- [Getting Started](#getting-started)
- [Model Distillation](#model-distillation)
- [Model Offerings](#model-offerings)
- [Examples](#examples)


Expand All @@ -38,6 +39,20 @@ The process of model distillation is a two stage process as seen below.
2. The second stage is finetuning. Once the synthetic data is collected, the student model is then finetuned off of the training and validation data created from the teacher model. This transfers the knowledge from the teacher model to the student model.


## Model Offerings
### Teacher Models
We currently support Meta Llama 3.1 405B Instruct as the teacher model for all distillation scenarios.

### Student Models
- Meta Llama 3.1 8B Instruct
- Phi 3 Mini 4k Instruct
- Phi 3 Mini 128k Instruct
- Phi 3 Medium 4k Instruct
- Phi 3 Medium 128k Instruct
- Phi 3.5 Mini Instruct
- Phi 3.5 MoE Instruct

We currently support Meta Llama 3.1 8B Instruct and all Microsoft Phi3 and Phi3.5 Instruct series models as student models. Fine-tuning of Meta Llama 3.1 Instruct series of models is only available in West US 3 region whereas Fine-tuning of Phi 3 and Phi 3.5 Instruct series of models is only available in East US 2 region. To use the distillation offering your workspace must be setup in the appropriate region for your selected student model.

## Examples
We currently support numerous task types for model distillation. To view examples on how to distill and consume a model with the SDK, click on the following task type of interest
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@
" \n",
"**Note :**\n",
" \n",
"- Distillation offering is only available in **West US 3** regions.\n",
"- Distillation should only be used for single turn chat completion format.\n",
"- The Meta Llama 3.1 405B Instruct model can only be used as a teacher model.\n",
"- The Meta Llama 3.1 8B Instruct can only be used as a student (target) model.\n",
"- Distillation of a Meta Llama 3.1 8B Instruct student (target) model is only available in **West US 3** regions.\n",
"- Distillation of Phi3 or Phi3.5 student (target) models is only available in **East US 2** regions.\n",
"\n",
"**Prerequisites :**\n",
"- Subscribe to the Meta Llama 3.1 405B Instruct and Meta Llama 3.1 8B Instruct, see [how to subscribe your project to the model offering in MS Learn](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-serverless?tabs=azure-ai-studio#subscribe-your-project-to-the-model-offering)"
Expand Down Expand Up @@ -94,7 +94,9 @@
"\n",
"### 1.3.1 Prerequisites\n",
"\n",
"An AI Studio project in **West US 3** is required. Please follow [this](https://learn.microsoft.com/azure/ai-studio/how-to/fine-tune-model-llama?tabs=llama-two%2Cchatcompletion#prerequisites) document to setup your AI Studio project\n",
"For distillation of a Meta Llama 3.1 8B student model, an AI Studio project in **West US 3** is required. Please follow [this](https://learn.microsoft.com/azure/ai-studio/how-to/fine-tune-model-llama?tabs=llama-two%2Cchatcompletion#prerequisites) document to setup your AI Studio project\n",
"\n",
"If you are using a Phi 3 or Phi 3.5 student model, an AI Studio project in **East US 2** is required. Follow [this](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/fine-tune-phi-3?tabs=phi-3-mini#prerequisites) document to setup your AI Studio project\n",
"\n",
"### 1.3.2 AI Studio project settings\n",
"\n",
Expand Down Expand Up @@ -404,7 +406,13 @@
"source": [
"#### Student Model\n",
"Select the student model to use. Supported student models:\n",
"1. Meta-Llama-3.1-8B-Instruct"
"1. Meta-Llama-3.1-8B-Instruct\n",
"2. Phi-3-Mini-4k-Instruct\n",
"3. Phi-3-Mini-128k-Instruct\n",
"4. Phi-3-Medium-4k-Instruct\n",
"5. Phi-3-Medium-128k-Instruct\n",
"6. Phi-3.5-Mini-Instruct\n",
"7. Phi-3.5-MoE-Instruct"
]
},
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@
" \n",
"**Note :**\n",
" \n",
"- Distillation offering is only available in **West US 3** regions.\n",
"- Distillation should only be used for single turn chat completion format.\n",
"- The Meta Llama 3.1 405B Instruct model can only be used as a teacher model.\n",
"- The Meta Llama 3.1 8B Instruct can only be used as a student (target) model.\n",
"- Distillation of a Meta Llama 3.1 8B Instruct student (target) model is only available in **West US 3** regions.\n",
"- Distillation of Phi3 or Phi3.5 student (target) models is only available in **East US 2** regions.\n",
"\n",
"**Prerequisites :**\n",
"- Subscribe to the Meta Llama 3.1 405B Instruct and Meta Llama 3.1 8B Instruct, see [how to subscribe your project to the model offering in MS Learn](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-serverless?tabs=azure-ai-studio#subscribe-your-project-to-the-model-offering)"
Expand Down Expand Up @@ -98,7 +98,9 @@
"\n",
"### 1.3.1 Prerequisites\n",
"\n",
"An AI Studio project in **West US 3** is required. Please follow [this](https://learn.microsoft.com/azure/ai-studio/how-to/fine-tune-model-llama?tabs=llama-two%2Cchatcompletion#prerequisites) document to setup your AI Studio project\n",
"For distillation of a Meta Llama 3.1 8B student model, an AI Studio project in **West US 3** is required. Please follow [this](https://learn.microsoft.com/azure/ai-studio/how-to/fine-tune-model-llama?tabs=llama-two%2Cchatcompletion#prerequisites) document to setup your AI Studio project\n",
"\n",
"If you are using a Phi 3 or Phi 3.5 student model, an AI Studio project in **East US 2** is required. Follow [this](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/fine-tune-phi-3?tabs=phi-3-mini#prerequisites) document to setup your AI Studio project\n",
"\n",
"### 1.3.2 AI Studio project settings\n",
"\n",
Expand Down Expand Up @@ -400,7 +402,13 @@
"source": [
"#### Student Model\n",
"Select the student model to use. Supported student models:\n",
"1. Meta-Llama-3.1-8B-Instruct"
"1. Meta-Llama-3.1-8B-Instruct\n",
"2. Phi-3-Mini-4k-Instruct\n",
"3. Phi-3-Mini-128k-Instruct\n",
"4. Phi-3-Medium-4k-Instruct\n",
"5. Phi-3-Medium-128k-Instruct\n",
"6. Phi-3.5-Mini-Instruct\n",
"7. Phi-3.5-MoE-Instruct"
]
},
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@
" \n",
"**Note :**\n",
" \n",
"- Distillation offering is only available in **West US 3** regions.\n",
"- Distillation should only be used for single turn chat completion format.\n",
"- The Meta Llama 3.1 405B Instruct model can only be used as a teacher model.\n",
"- The Meta Llama 3.1 8B Instruct can only be used as a student (target) model.\n",
"- Distillation of a Meta Llama 3.1 8B Instruct student (target) model is only available in **West US 3** regions.\n",
"- Distillation of Phi3 or Phi3.5 student (target) models is only available in **East US 2** regions.\n",
"\n",
"**Prerequisites :**\n",
"- Subscribe to the Meta Llama 3.1 405B Instruct and Meta Llama 3.1 8B Instruct, see [how to subscribe your project to the model offering in MS Learn](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-serverless?tabs=azure-ai-studio#subscribe-your-project-to-the-model-offering)"
Expand Down Expand Up @@ -97,7 +97,9 @@
"\n",
"### 1.3.1 Prerequisites\n",
"\n",
"An AI Studio project in **West US 3** is required. Please follow [this](https://learn.microsoft.com/azure/ai-studio/how-to/fine-tune-model-llama?tabs=llama-two%2Cchatcompletion#prerequisites) document to setup your AI Studio project\n",
"For distillation of a Meta Llama 3.1 8B student model, an AI Studio project in **West US 3** is required. Please follow [this](https://learn.microsoft.com/azure/ai-studio/how-to/fine-tune-model-llama?tabs=llama-two%2Cchatcompletion#prerequisites) document to setup your AI Studio project\n",
"\n",
"If you are using a Phi 3 or Phi 3.5 student model, an AI Studio project in **East US 2** is required. Follow [this](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/fine-tune-phi-3?tabs=phi-3-mini#prerequisites) document to setup your AI Studio project\n",
"\n",
"### 1.3.2 AI Studio project settings\n",
"\n",
Expand Down Expand Up @@ -399,7 +401,13 @@
"source": [
"#### Student Model\n",
"Select the student model to use by using the model id of the model in the model catalog. Supported student models:\n",
"1. Meta-Llama-3.1-8B-Instruct"
"1. Meta-Llama-3.1-8B-Instruct\n",
"2. Phi-3-Mini-4k-Instruct\n",
"3. Phi-3-Mini-128k-Instruct\n",
"4. Phi-3-Medium-4k-Instruct\n",
"5. Phi-3-Medium-128k-Instruct\n",
"6. Phi-3.5-Mini-Instruct\n",
"7. Phi-3.5-MoE-Instruct"
]
},
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,10 +14,10 @@
" \n",
"**Note :**\n",
" \n",
"- Distillation offering is only available in **West US 3** regions.\n",
"- Distillation should only be used for single turn chat completion format.\n",
"- The Meta Llama 3.1 405B Instruct model can only be used as a teacher model.\n",
"- The Meta Llama 3.1 8B Instruct can only be used as a student (target) model.\n",
"- Distillation of a Meta Llama 3.1 8B Instruct student (target) model is only available in **West US 3** regions.\n",
"- Distillation of Phi3 or Phi3.5 student (target) models is only available in **East US 2** regions.\n",
"\n",
"**Prerequisites :**\n",
"- Subscribe to the Meta Llama 3.1 405B Instruct and Meta Llama 3.1 8B Instruct, see [how to subscribe your project to the model offering in MS Learn](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-serverless?tabs=azure-ai-studio#subscribe-your-project-to-the-model-offering)"
Expand Down Expand Up @@ -94,7 +94,9 @@
"\n",
"### 1.3.1 Prerequisites\n",
"\n",
"An AI Studio project in **West US 3** is required. Please follow [this](https://learn.microsoft.com/azure/ai-studio/how-to/fine-tune-model-llama?tabs=llama-two%2Cchatcompletion#prerequisites) document to setup your AI Studio project\n",
"For distillation of a Meta Llama 3.1 8B student model, an AI Studio project in **West US 3** is required. Please follow [this](https://learn.microsoft.com/azure/ai-studio/how-to/fine-tune-model-llama?tabs=llama-two%2Cchatcompletion#prerequisites) document to setup your AI Studio project\n",
"\n",
"If you are using a Phi 3 or Phi 3.5 student model, an AI Studio project in **East US 2** is required. Follow [this](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/fine-tune-phi-3?tabs=phi-3-mini#prerequisites) document to setup your AI Studio project\n",
"\n",
"### 1.3.2 AI Studio project settings\n",
"\n",
Expand Down Expand Up @@ -404,7 +406,13 @@
"source": [
"#### Student Model\n",
"Select the student model to use. Supported student models:\n",
"1. Meta-Llama-3.1-8B-Instruct"
"1. Meta-Llama-3.1-8B-Instruct\n",
"2. Phi-3-Mini-4k-Instruct\n",
"3. Phi-3-Mini-128k-Instruct\n",
"4. Phi-3-Medium-4k-Instruct\n",
"5. Phi-3-Medium-128k-Instruct\n",
"6. Phi-3.5-Mini-Instruct\n",
"7. Phi-3.5-MoE-Instruct"
]
},
{
Expand Down
Original file line number Diff line number Diff line change
Expand Up @@ -14,7 +14,6 @@
" \n",
"**Note :**\n",
" \n",
"- Distillation offering is only available in **West US 3** regions.\n",
"- Distillation should only be used for single turn chat completion format as shown below\n",
" ```json\n",
" {\"messages\": [\n",
Expand All @@ -23,7 +22,8 @@
" ]}\n",
" ```\n",
"- The Meta Llama 3.1 405B Instruct model can only be used as a teacher model.\n",
"- The Meta Llama 3.1 8B Instruct can only be used as a student (target) model.\n",
"- Distillation of a Meta Llama 3.1 8B Instruct student (target) model is only available in **West US 3** regions.\n",
"- Distillation of Phi3 or Phi3.5 student (target) models is only available in **East US 2** regions.\n",
"\n",
"**Prerequisites :**\n",
"- Subscribe to the Meta Llama 3.1 405B Instruct and Meta Llama 3.1 8B Instruct, see [how to subscribe your project to the model offering in MS Learn](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/deploy-models-serverless?tabs=azure-ai-studio#subscribe-your-project-to-the-model-offering)"
Expand Down Expand Up @@ -104,7 +104,9 @@
"\n",
"### 1.3.1 Prerequisites\n",
"\n",
"An AI Studio project in **West US 3** is required. Please follow [this](https://learn.microsoft.com/azure/ai-studio/how-to/fine-tune-model-llama?tabs=llama-two%2Cchatcompletion#prerequisites) document to setup your AI Studio project\n",
"For distillation of a Meta Llama 3.1 8B student model, an AI Studio project in **West US 3** is required. Please follow [this](https://learn.microsoft.com/azure/ai-studio/how-to/fine-tune-model-llama?tabs=llama-two%2Cchatcompletion#prerequisites) document to setup your AI Studio project\n",
"\n",
"If you are using a Phi 3 or Phi 3.5 student model, an AI Studio project in **East US 2** is required. Follow [this](https://learn.microsoft.com/en-us/azure/ai-studio/how-to/fine-tune-phi-3?tabs=phi-3-mini#prerequisites) document to setup your AI Studio project\n",
"\n",
"### 1.3.2 AI Studio project settings\n",
"\n",
Expand Down Expand Up @@ -414,7 +416,13 @@
"source": [
"#### Student Model\n",
"Select the student model to use. Supported student models:\n",
"1. Meta-Llama-3.1-8B-Instruct"
"1. Meta-Llama-3.1-8B-Instruct\n",
"2. Phi-3-Mini-4k-Instruct\n",
"3. Phi-3-Mini-128k-Instruct\n",
"4. Phi-3-Medium-4k-Instruct\n",
"5. Phi-3-Medium-128k-Instruct\n",
"6. Phi-3.5-Mini-Instruct\n",
"7. Phi-3.5-MoE-Instruct"
]
},
{
Expand Down

0 comments on commit 2a949a4

Please sign in to comment.