Skip to content

Commit

Permalink
Notebook/torchvision notebook link (#1174)
Browse files Browse the repository at this point in the history 
Update torchvision notebook name
---------

Co-authored-by: yardeny-sony <yardeny-sony@sony.com>
  • Loading branch information
@yarden-yagil-sony
yarden-yagil-sony and yardeny-sony authored Aug 19, 2024
1 parent 1ecb55d commit 103ccb5
Showing 1 changed file with 293 additions and 0 deletions.
293 changes: 293 additions & 0 deletions ...ebooks/imx500_notebooks/pytorch/pytorch_torchvision_classification_model_for_imx500.ipynb
View file
Original file line number Diff line number Diff line change
@@ -0,0 +1,293 @@
{
"cells": [
{
"metadata": {},
"cell_type": "markdown",
"source": [
"# PyTorch Model from torchvision - Quantization for IMX500\n",
"\n",
"[Run this tutorial in Google Colab](https://colab.research.google.com/github/sony/model_optimization/blob/main/tutorials/notebooks/imx500_notebooks/pytorch/pytorch_torchvision_classification_model_for_imx500.ipynb)\n",
"\n",
"## Overview\n",
"\n",
"In this tutorial, we will illustrate a basic and quick process of preparing a pre-trained model for deployment using MCT. \n",
"We will use an existing pre-trained model from [torchvision](https://pytorch.org/vision/stable/models.html). The user can choose any torchvision model from this list. "
],
"id": "ca9e0ba1f92d22fc"
},
{
"metadata": {},
"cell_type": "markdown",
"source": [
"## Setup\n",
"### Install the relevant packages"
],
"id": "7e737ed9d9b11d2a"
},
{
"metadata": {},
"cell_type": "code",
"source": [
"!pip install -q torch\n",
"!pip install -q torchvision\n",
"!pip install -q onnx"
],
"id": "5250b07975f15b5f",
"outputs": [],
"execution_count": null
},
{
"metadata": {},
"cell_type": "markdown",
"source": "Install MCT (if it’s not already installed). Additionally, in order to use all the necessary utility functions for this tutorial, we also copy [MCT tutorials folder](https://github.com/sony/model_optimization/tree/main/tutorials) and add it to the system path.",
"id": "b1a05efedd4dbc77"
},
{
"metadata": {},
"cell_type": "code",
"source": [
"import sys\n",
"import importlib\n",
"\n",
"if not importlib.util.find_spec('model_compression_toolkit'):\n",
" !pip install model_compression_toolkit\n",
"!git clone https://github.com/sony/model_optimization.git temp_mct && mv temp_mct/tutorials . && \\rm -rf temp_mct\n",
"sys.path.insert(0,\"tutorials\")"
],
"id": "391a9ed0d178002e",
"outputs": [],
"execution_count": null
},
{
"metadata": {},
"cell_type": "markdown",
"source": [
"### Download ImageNet validation set\n",
"Download ImageNet dataset with only the validation split.\n",
"\n",
"Note that for demonstration purposes we use the validation set for the model quantization routines. Usually, a subset of the training dataset is used, but loading it is a heavy procedure that is unnecessary for the sake of this demonstration.\n",
"\n",
"This step may take several minutes..."
],
"id": "5923827fab97d1de"
},
{
"metadata": {},
"cell_type": "code",
"source": [
"import os\n",
"if not os.path.isdir('imagenet'):\n",
" !mkdir imagenet\n",
" !wget https://image-net.org/data/ILSVRC/2012/ILSVRC2012_devkit_t12.tar.gz\n",
" !mv ILSVRC2012_devkit_t12.tar.gz imagenet/\n",
" !wget https://image-net.org/data/ILSVRC/2012/ILSVRC2012_img_val.tar\n",
" !mv ILSVRC2012_img_val.tar imagenet/"
],
"id": "ec301c30cd83d535",
"outputs": [],
"execution_count": null
},
{
"metadata": {},
"cell_type": "markdown",
"source": [
"## Model Quantization\n",
"\n",
"### Download a Pre-Trained Model"
],
"id": "7059e58ac6efff74"
},
{
"metadata": {},
"cell_type": "code",
"source": [
"# from torchvision.models import mobilenet_v2, MobileNet_V2_Weights\n",
"from torchvision.models import efficientnet_b0, EfficientNet_B0_Weights\n",
"\n",
"model = efficientnet_b0(weights=EfficientNet_B0_Weights)\n",
"model.eval()"
],
"id": "ea84e114c819dde0",
"outputs": [],
"execution_count": null
},
{
"metadata": {},
"cell_type": "markdown",
"source": [
"### Post training quantization using Model Compression Toolkit \n",
"\n",
"Now, we're all set to use MCT's post-training quantization. To begin, we'll define a representative dataset and proceed with the model quantization. Please note that, for demonstration purposes, we'll use the evaluation dataset as our representative dataset. We'll calibrate the model using 80 representative images, divided into 20 iterations of 'batch_size' images each. "
],
"id": "7d334e3dd747ba68"
},
{
"metadata": {},
"cell_type": "code",
"source": [
"import model_compression_toolkit as mct\n",
"from model_compression_toolkit.core.pytorch.pytorch_device_config import get_working_device\n",
"from typing import Iterator, Tuple, List\n",
"from torch.utils.data import DataLoader\n",
"from torchvision import transforms, datasets\n",
"\n",
"\n",
"BATCH_SIZE = 4\n",
"n_iters = 20\n",
"device = get_working_device()\n",
"\n",
"# Define transformations for the validation set\n",
"val_transform = transforms.Compose([\n",
" transforms.Resize(256),\n",
" transforms.CenterCrop(224),\n",
" transforms.ToTensor(),\n",
" transforms.Normalize(mean=[0.485, 0.456, 0.406], std=[0.229, 0.224, 0.225])])\n",
"\n",
"# Extract ImageNet validation dataset using torchvision \"datasets\" module\n",
"val_dataset = datasets.ImageNet(root='./imagenet', split='val', transform=val_transform)\n",
"val_loader = DataLoader(val_dataset, batch_size=BATCH_SIZE, shuffle=False, num_workers=4)\n",
"\n",
"# Define representative dataset generator\n",
"def get_representative_dataset(n_iter: int, dataset_loader: Iterator[Tuple]):\n",
" \"\"\"\n",
" This function creates a representative dataset generator. The generator yields numpy\n",
" arrays of batches of shape: [Batch, H, W ,C].\n",
" Args:\n",
" n_iter: number of iterations for MCT to calibrate on\n",
" Returns:\n",
" A representative dataset generator\n",
" \"\"\" \n",
" def representative_dataset() -> Iterator[List]:\n",
" ds_iter = iter(dataset_loader)\n",
" for _ in range(n_iter):\n",
" yield [next(ds_iter)[0]]\n",
"\n",
" return representative_dataset\n",
"\n",
"# Get representative dataset generator\n",
"representative_dataset_gen = get_representative_dataset(n_iter=n_iters, dataset_loader=val_loader)\n",
"\n",
"# Perform post training quantization with the default configuration\n",
"quant_model, _ = mct.ptq.pytorch_post_training_quantization(model, representative_data_gen=representative_dataset_gen)\n",
"print('Quantized model is ready')"
],
"id": "c231499204b5fe58",
"outputs": [],
"execution_count": null
},
{
"metadata": {},
"cell_type": "markdown",
"source": [
"### Model Export\n",
"\n",
"Now, we can export the quantized model, ready for deployment, into a `.onnx` format file. Please ensure that the `save_model_path` has been set correctly. "
],
"id": "2659cdc8ec1a3008"
},
{
"metadata": {},
"cell_type": "code",
"source": [
"mct.exporter.pytorch_export_model(model=quant_model,\n",
" save_model_path='./torchvision_qmodel.onnx',\n",
" repr_dataset=representative_dataset_gen,\n",
" onnx_opset_version=17)"
],
"id": "de8dd4dbc0c8d393",
"outputs": [],
"execution_count": null
},
{
"metadata": {},
"cell_type": "markdown",
"source": [
"## Evaluation on ImageNet dataset\n",
"\n",
"### Floating point model evaluation\n",
"Please ensure that the dataset path has been set correctly before running this code cell."
],
"id": "d98c5b8acb2e7397"
},
{
"metadata": {},
"cell_type": "code",
"source": [
"from tutorials.resources.utils.pytorch_tutorial_tools import classification_eval\n",
"# Evaluate the model on ImageNet\n",
"eval_results = classification_eval(model, val_loader)\n",
"\n",
"# Print float model Accuracy results\n",
"print(\"Float model Accuracy: {:.4f}\".format(round(100 * eval_results[0], 2)))"
],
"id": "6cd077108ef55889",
"outputs": [],
"execution_count": null
},
{
"metadata": {},
"cell_type": "markdown",
"source": [
"### Quantized model evaluation\n",
"We can evaluate the performance of the quantized model. There is a slight decrease in performance that can be further mitigated by either expanding the representative dataset or employing MCT's advanced quantization methods, such as GPTQ (Gradient-Based/Enhanced Post Training Quantization)."
],
"id": "eb86aae997a5210a"
},
{
"metadata": {},
"cell_type": "code",
"source": [
"# Evaluate the quantized model on ImageNet\n",
"eval_results = classification_eval(quant_model, val_loader)\n",
"\n",
"# Print quantized model Accuracy results\n",
"print(\"Quantized model Accuracy: {:.4f}\".format(round(100 * eval_results[0], 2)))"
],
"id": "b881138870761d89",
"outputs": [],
"execution_count": null
},
{
"metadata": {},
"cell_type": "markdown",
"source": [
"\\\n",
"Copyright 2024 Sony Semiconductor Israel, Inc. All rights reserved.\n",
"\n",
"Licensed under the Apache License, Version 2.0 (the \"License\");\n",
"you may not use this file except in compliance with the License.\n",
"You may obtain a copy of the License at\n",
"\n",
" http://www.apache.org/licenses/LICENSE-2.0\n",
"\n",
"Unless required by applicable law or agreed to in writing, software\n",
"distributed under the License is distributed on an \"AS IS\" BASIS,\n",
"WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied.\n",
"See the License for the specific language governing permissions and\n",
"limitations under the License"
],
"id": "a4d344795b87475a"
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 2
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython2",
"version": "2.7.6"
}
},
"nbformat": 4,
"nbformat_minor": 5
}

0 comments on commit 103ccb5

Please sign in to comment.