Torchvision tutorial notebook #1171
There are no files selected for viewing
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
This file contains bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
| Original file line number | Diff line number | Diff line change |
|---|---|---|
| @@ -0,0 +1,293 @@ | ||
| { | ||
|
There was a problem hiding this comment. clear section outputs before commiting the file. |
||
| "cells": [ | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "# PyTorch Model from torchvision - Quantization for IMX500\n", | ||
| "\n", | ||
| "[Run this tutorial in Google Colab](https://colab.research.google.com/github/sony/model_optimization/blob/main/tutorials/notebooks/imx500_notebooks/pytorch/pytorch_torchvision_classification_model_for_imx500.ipynb)\n", | ||
| "\n", | ||
| "## Overview\n", | ||
| "\n", | ||
| "In this tutorial, we will illustrate a basic and quick process of preparing a pre-trained model for deployment using MCT. \n", | ||
| "We will use an existing pre-trained model from [torchvision](https://pytorch.org/vision/stable/models.html). The user can choose any torchvision model from this list. " | ||
| ], | ||
| "id": "ca9e0ba1f92d22fc" | ||
| }, | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "## Setup\n", | ||
| "### Install the relevant packages" | ||
| ], | ||
| "id": "7e737ed9d9b11d2a" | ||
| }, | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "!pip install -q torch\n", | ||
| "!pip install -q torchvision\n", | ||
| "!pip install -q onnx" | ||
| ], | ||
| "id": "5250b07975f15b5f", | ||
| "outputs": [], | ||
| "execution_count": null | ||
| }, | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "markdown", | ||
| "source": "Install MCT (if it’s not already installed). Additionally, in order to use all the necessary utility functions for this tutorial, we also copy [MCT tutorials folder](https://github.com/sony/model_optimization/tree/main/tutorials) and add it to the system path.", | ||
| "id": "b1a05efedd4dbc77" | ||
| }, | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "import sys\n", | ||
| "import importlib\n", | ||
| "\n", | ||
| "if not importlib.util.find_spec('model_compression_toolkit'):\n", | ||
| " !pip install model_compression_toolkit\n", | ||
| "!git clone https://github.com/sony/model_optimization.git temp_mct && mv temp_mct/tutorials . && \\rm -rf temp_mct\n", | ||
| "sys.path.insert(0,\"tutorials\")" | ||
| ], | ||
| "id": "391a9ed0d178002e", | ||
| "outputs": [], | ||
| "execution_count": null | ||
| }, | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "### Download ImageNet validation set\n", | ||
| "Download ImageNet dataset with only the validation split.\n", | ||
| "\n", | ||
| "Note that for demonstration purposes we use the validation set for the model quantization routines. Usually, a subset of the training dataset is used, but loading it is a heavy procedure that is unnecessary for the sake of this demonstration.\n", | ||
| "\n", | ||
| "This step may take several minutes..." | ||
| ], | ||
| "id": "5923827fab97d1de" | ||
| }, | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "import os\n", | ||
| "if not os.path.isdir('imagenet'):\n", | ||
| " !mkdir imagenet\n", | ||
| " !wget https://image-net.org/data/ILSVRC/2012/ILSVRC2012_devkit_t12.tar.gz\n", | ||
| " !mv ILSVRC2012_devkit_t12.tar.gz imagenet/\n", | ||
| " !wget https://image-net.org/data/ILSVRC/2012/ILSVRC2012_img_val.tar\n", | ||
| " !mv ILSVRC2012_img_val.tar imagenet/" | ||
| ], | ||
| "id": "ec301c30cd83d535", | ||
| "outputs": [], | ||
| "execution_count": null | ||
| }, | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "## Model Quantization\n", | ||
| "\n", | ||
| "### Download a Pre-Trained Model" | ||
| ], | ||
| "id": "7059e58ac6efff74" | ||
| }, | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "# from torchvision.models import mobilenet_v2, MobileNet_V2_Weights\n", | ||
| "from torchvision.models import efficientnet_b0, EfficientNet_B0_Weights\n", | ||
| "\n", | ||
| "model = efficientnet_b0(weights=EfficientNet_B0_Weights)\n", | ||
| "model.eval()" | ||
| ], | ||
| "id": "ea84e114c819dde0", | ||
| "outputs": [], | ||
| "execution_count": null | ||
| }, | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "### Post training quantization using Model Compression Toolkit \n", | ||
| "\n", | ||
| "Now, we're all set to use MCT's post-training quantization. To begin, we'll define a representative dataset and proceed with the model quantization. Please note that, for demonstration purposes, we'll use the evaluation dataset as our representative dataset. We'll calibrate the model using 80 representative images, divided into 20 iterations of 'batch_size' images each. " | ||
| ], | ||
| "id": "7d334e3dd747ba68" | ||
| }, | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "import model_compression_toolkit as mct\n", | ||
| "from model_compression_toolkit.core.pytorch.pytorch_device_config import get_working_device\n", | ||
| "from typing import Iterator, Tuple, List\n", | ||
| "from torch.utils.data import DataLoader\n", | ||
| "from torchvision import transforms, datasets\n", | ||
| "\n", | ||
| "\n", | ||
| "BATCH_SIZE = 4\n", | ||
| "n_iters = 20\n", | ||
| "device = get_working_device()\n", | ||
| "\n", | ||
| "# Define transformations for the validation set\n", | ||
| "val_transform = transforms.Compose([\n", | ||
| " transforms.Resize(256),\n", | ||
| " transforms.CenterCrop(224),\n", | ||
| " transforms.ToTensor(),\n", | ||
| " transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])])\n", | ||
| "\n", | ||
| "# Extract ImageNet validation dataset using torchvision \"datasets\" module\n", | ||
| "val_dataset = datasets.ImageNet(root='./imagenet', split='val', transform=val_transform)\n", | ||
| "val_loader = DataLoader(val_dataset, batch_size=BATCH_SIZE, shuffle=False, num_workers=4)\n", | ||
| "\n", | ||
| "# Define representative dataset generator\n", | ||
| "def get_representative_dataset(n_iter: int, dataset_loader: Iterator[Tuple]):\n", | ||
| " \"\"\"\n", | ||
| " This function creates a representative dataset generator. The generator yields numpy\n", | ||
| " arrays of batches of shape: [Batch, H, W ,C].\n", | ||
| " Args:\n", | ||
| " n_iter: number of iterations for MCT to calibrate on\n", | ||
| " Returns:\n", | ||
| " A representative dataset generator\n", | ||
| " \"\"\" \n", | ||
| " def representative_dataset() -> Iterator[List]:\n", | ||
| " ds_iter = iter(dataset_loader)\n", | ||
| " for _ in range(n_iter):\n", | ||
| " yield [next(ds_iter)[0]]\n", | ||
| "\n", | ||
| " return representative_dataset\n", | ||
| "\n", | ||
| "# Get representative dataset generator\n", | ||
| "representative_dataset_gen = get_representative_dataset(n_iter=n_iters, dataset_loader=val_loader)\n", | ||
| "\n", | ||
| "# Perform post training quantization with the default configuration\n", | ||
| "quant_model, _ = mct.ptq.pytorch_post_training_quantization(model, representative_data_gen=representative_dataset_gen)\n", | ||
| "print('Quantized model is ready')" | ||
| ], | ||
| "id": "c231499204b5fe58", | ||
| "outputs": [], | ||
| "execution_count": null | ||
| }, | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "### Model Export\n", | ||
| "\n", | ||
| "Now, we can export the quantized model, ready for deployment, into a `.onnx` format file. Please ensure that the `save_model_path` has been set correctly. " | ||
| ], | ||
| "id": "2659cdc8ec1a3008" | ||
| }, | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "mct.exporter.pytorch_export_model(model=quant_model,\n", | ||
| " save_model_path='./torchvision_qmodel.onnx',\n", | ||
| " repr_dataset=representative_dataset_gen,\n", | ||
| " onnx_opset_version=17)" | ||
| ], | ||
| "id": "de8dd4dbc0c8d393", | ||
| "outputs": [], | ||
| "execution_count": null | ||
| }, | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "## Evaluation on ImageNet dataset\n", | ||
| "\n", | ||
| "### Floating point model evaluation\n", | ||
| "Please ensure that the dataset path has been set correctly before running this code cell." | ||
| ], | ||
| "id": "d98c5b8acb2e7397" | ||
| }, | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "from tutorials.resources.utils.pytorch_tutorial_tools import classification_eval\n", | ||
| "# Evaluate the model on ImageNet\n", | ||
| "eval_results = classification_eval(model, val_loader)\n", | ||
| "\n", | ||
| "# Print float model Accuracy results\n", | ||
| "print(\"Float model Accuracy: {:.4f}\".format(round(100 * eval_results[0], 2)))" | ||
| ], | ||
| "id": "6cd077108ef55889", | ||
| "outputs": [], | ||
| "execution_count": null | ||
| }, | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "### Quantized model evaluation\n", | ||
| "We can evaluate the performance of the quantized model. There is a slight decrease in performance that can be further mitigated by either expanding the representative dataset or employing MCT's advanced quantization methods, such as GPTQ (Gradient-Based/Enhanced Post Training Quantization)." | ||
| ], | ||
| "id": "eb86aae997a5210a" | ||
| }, | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "code", | ||
| "source": [ | ||
| "# Evaluate the quantized model on ImageNet\n", | ||
| "eval_results = classification_eval(quant_model, val_loader)\n", | ||
| "\n", | ||
| "# Print quantized model Accuracy results\n", | ||
| "print(\"Quantized model Accuracy: {:.4f}\".format(round(100 * eval_results[0], 2)))" | ||
| ], | ||
| "id": "b881138870761d89", | ||
| "outputs": [], | ||
| "execution_count": null | ||
| }, | ||
| { | ||
| "metadata": {}, | ||
| "cell_type": "markdown", | ||
| "source": [ | ||
| "\\\n", | ||
| "Copyright 2024 Sony Semiconductor Israel, Inc. All rights reserved.\n", | ||
| "\n", | ||
| "Licensed under the Apache License, Version 2.0 (the \"License\");\n", | ||
| "you may not use this file except in compliance with the License.\n", | ||
| "You may obtain a copy of the License at\n", | ||
| "\n", | ||
| " http://www.apache.org/licenses/LICENSE-2.0\n", | ||
| "\n", | ||
| "Unless required by applicable law or agreed to in writing, software\n", | ||
| "distributed under the License is distributed on an \"AS IS\" BASIS,\n", | ||
| "WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n", | ||
| "See the License for the specific language governing permissions and\n", | ||
| "limitations under the License" | ||
| ], | ||
| "id": "a4d344795b87475a" | ||
| } | ||
| ], | ||
| "metadata": { | ||
| "kernelspec": { | ||
| "display_name": "Python 3", | ||
| "language": "python", | ||
| "name": "python3" | ||
| }, | ||
| "language_info": { | ||
| "codemirror_mode": { | ||
| "name": "ipython", | ||
| "version": 2 | ||
| }, | ||
| "file_extension": ".py", | ||
| "mimetype": "text/x-python", | ||
| "name": "python", | ||
| "nbconvert_exporter": "python", | ||
| "pygments_lexer": "ipython2", | ||
| "version": "2.7.6" | ||
| } | ||
| }, | ||
| "nbformat": 4, | ||
| "nbformat_minor": 5 | ||
| } | ||
Oops, something went wrong.
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
confirm the link - it contains .... torchvision.models.RegNet_Y_400MF_Weights...