From d404633432f4d30ff0d12b4f20e01429853cc93c Mon Sep 17 00:00:00 2001 From: Ammar Ahmad Awan Date: Thu, 1 Apr 2021 13:13:33 -0700 Subject: [PATCH 1/4] Add pointer to AML examples. --- docs/_tutorials/getting-started.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/_tutorials/getting-started.md b/docs/_tutorials/getting-started.md index e9b9aa0e627e..ecd3159df8c9 100644 --- a/docs/_tutorials/getting-started.md +++ b/docs/_tutorials/getting-started.md @@ -9,6 +9,7 @@ date: 2020-05-15 * Installing is as simple as `pip install deepspeed`, [see more details](/tutorials/advanced-install/). * Please see our [Azure tutorial](/tutorials/azure/) to get started with DeepSpeed on Azure! +* To get started with DeepSpeed on AzureML, please see the [AzureML Examples GitHub](https://github.com/Azure/azureml-examples/tree/main/workflows/train/deepspeed) * If you're not on Azure, we recommend using our docker image via `docker pull deepspeed/deepspeed:latest` which contains a pre-installed version of DeepSpeed and all the necessary dependencies. ## Writing DeepSpeed Models From c20e120ffb13ee38dcb4f8d97f8068430adc7ec7 Mon Sep 17 00:00:00 2001 From: Ammar Ahmad Awan Date: Fri, 2 Apr 2021 13:18:10 -0700 Subject: [PATCH 2/4] Add AML pointer in news items --- README.md | 1 + 1 file changed, 1 insertion(+) diff --git a/README.md b/README.md index f96808bd9cad..c7bde12dd0ea 100755 --- a/README.md +++ b/README.md @@ -33,6 +33,7 @@ information [here](https://innovation.microsoft.com/en-us/exploring-ai-at-scale) # News +* [2021/04/01] [[DeepSpeed on AzureML] Transformers and CIFAR examples are now available on AzureML GitHub](https://github.com/Azure/azureml-examples/tree/main/workflows/train/deepspeed) * [2021/03/30] [[PyTorch Lightning Blog] Accessible Multi-Billion Parameter Model Training with PyTorch Lightning + DeepSpeed](https://medium.com/pytorch-lightning/accessible-multi-billion-parameter-model-training-with-pytorch-lightning-deepspeed-c9333ac3bb59) * [2021/03/16] [1-bit Adam v2: NCCL-based implementation and more](https://www.deepspeed.ai/tutorials/onebit-adam/) * [2021/03/08] [ZeRO-3 Offload: Scale your models to trillion parameters without code changes while leveraging both CPUs & GPUs](https://www.deepspeed.ai/news/2021/03/07/zero3-offload.html) From feb1e1cd84ed568e3a9aa9aefe171d41c05a3d11 Mon Sep 17 00:00:00 2001 From: Ammar Ahmad Awan Date: Fri, 2 Apr 2021 13:19:50 -0700 Subject: [PATCH 3/4] add aml pointer in news --- docs/index.md | 1 + 1 file changed, 1 insertion(+) diff --git a/docs/index.md b/docs/index.md index 31755b8e3201..497f88bab5c3 100755 --- a/docs/index.md +++ b/docs/index.md @@ -30,6 +30,7 @@ initiative to enable next-generation AI capabilities at scale, where you can fin information [here](https://innovation.microsoft.com/en-us/exploring-ai-at-scale). # What's New? +* [2021/04/02] [[DeepSpeed on AzureML] Transformers and CIFAR examples are now available on AzureML GitHub](https://github.com/Azure/azureml-examples/tree/main/workflows/train/deepspeed) * [2021/03/30] [[PyTorch Lightning Blog] Accessible Multi-Billion Parameter Model Training with PyTorch Lightning + DeepSpeed](https://medium.com/pytorch-lightning/accessible-multi-billion-parameter-model-training-with-pytorch-lightning-deepspeed-c9333ac3bb59) * [2021/03/16] [1-bit Adam v2: NCCL-based implementation and more](https://www.deepspeed.ai/tutorials/onebit-adam/) * [2021/03/08] [ZeRO-3 Offload: Scale your models to trillion parameters without code changes while leveraging both CPUs & GPUs](https://www.deepspeed.ai/news/2021/03/07/zero3-offload.html) From f70baa9406dcd965640e1865685cb3f38b3eb138 Mon Sep 17 00:00:00 2001 From: Ammar Ahmad Awan Date: Fri, 2 Apr 2021 13:22:22 -0700 Subject: [PATCH 4/4] add aml pointer --- docs/_tutorials/azure.md | 2 ++ 1 file changed, 2 insertions(+) diff --git a/docs/_tutorials/azure.md b/docs/_tutorials/azure.md index 3644b4621f8f..45d41a618a23 100644 --- a/docs/_tutorials/azure.md +++ b/docs/_tutorials/azure.md @@ -10,6 +10,8 @@ benefit all your large model training jobs. If you don't already have an Azure account please see more details here: [https://azure.microsoft.com/](https://azure.microsoft.com/). +To use DeepSpeed on [Azure ML](https://azure.microsoft.com/en-us/services/machine-learning/), please take a look at easy-to-use examples for Transformers and CIFAR training from [AzureML Examples GitHub](https://github.com/Azure/azureml-examples/tree/main/workflows/train/deepspeed). + To help with launching Azure instances we suggest using the [Azure CLI](https://docs.microsoft.com/en-us/cli/azure/?view=azure-cli-latest). We have created several helper scripts to get you quickly started using DeepSpeed with Azure.