Iris-model-deployment-2024.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## XGBoost Model deployment in Amazon Sagemaker. \n",
    "\n",
    "#### This notebook should be run in an Amazon Sagemaker notebook instance. \n",
    "\n",
    "\n",
    "#### Before running this notebook, \n",
    "you should have uploaded the pre-trained model and test_point.csv from your laptop to the \n",
    "same folder where you have this notebook file. test_point.csv contains few sample test data in csv format.\n",
    "\n",
    "\n",
    "This loads the pre-trained XGBoost model and saves in a S3 bucket in .tar.gz format as required by Sagemaker.\n",
    "Then it creates a sagemaker model from the model file stored in S3. \n",
    "Then configures and creates an Endpoint to deploy the model and also tests invoking the endpoint to get prediction.\n",
    "\n",
    "#### Please remember not to run the last \"Delete the Endpoint\" cell if you want to test the deployed model from a client. \n",
    "\n",
    "\n",
    "After the exercise is over, \n",
    "##### you should cleanup the Sagemaker resources as described in \n",
    "https://docs.aws.amazon.com/sagemaker/latest/dg/ex1-cleanup.html to avoid charges incurred because of resources left behind.\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Import libraries"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "CPU times: user 259 ms, sys: 20.2 ms, total: 279 ms\n",
      "Wall time: 355 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "\n",
    "import os\n",
    "import boto3\n",
    "import sagemaker\n",
    "\n",
    "from sagemaker import get_execution_role\n",
    "\n",
    "region = boto3.Session().region_name\n",
    "\n",
    "role = get_execution_role()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create S3 bucket"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# This creates a default S3 bucket where we will upload our model.\n",
    "bucket = sagemaker.Session().default_bucket()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "bucket_path = \"https://s3-{}.amazonaws.com/{}\".format(region, bucket)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "arn:aws:iam::180494263813:role/service-role/AmazonSageMaker-ExecutionRole-20240703T092754\n",
      "us-east-1\n",
      "sagemaker-us-east-1-180494263813\n",
      "https://s3-us-east-1.amazonaws.com/sagemaker-us-east-1-180494263813\n"
     ]
    }
   ],
   "source": [
    "print(role)\n",
    "print(region)\n",
    "print(bucket)\n",
    "print(bucket_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Install xgboost as it is needed for loading the model from joblib dump file and test it before deployment.\n",
    "#### Please note that the XGBoost version should be same as the version with which the model was trained locally in laptop."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "#!conda install -y -c conda-forge xgboost==0.90"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "model_file_name = \"DEMO-local-xgboost-model\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Load the pre-trained model and test it before deployment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "import joblib\n",
    "import xgboost\n",
    "\n",
    "mymodel = joblib.load(model_file_name)\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 34,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "'1.7.6'"
      ]
     },
     "execution_count": 34,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "xgboost.__version__"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[5.4 3.  4.5 1.5]\n",
      " [5.6 3.  4.1 1.3]\n",
      " [6.3 2.8 5.1 1.5]\n",
      " [6.  3.  4.8 1.8]\n",
      " [5.1 3.3 1.7 0.5]]\n"
     ]
    }
   ],
   "source": [
    "#import json\n",
    "import numpy as np\n",
    "\n",
    "\n",
    "file_name = (\n",
    "    \"test_point.csv\"  # customize to your test file, will be 'mnist.single.test' if use data above\n",
    ")\n",
    "\n",
    "with open(file_name, \"r\") as f:\n",
    "    mypayload = np.loadtxt(f, delimiter=\",\")\n",
    "    \n",
    "print(mypayload)    "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([1, 1, 1, 2, 0], dtype=int32)"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "mymodel.predict(mypayload)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Create a tar.gz model file as this is the format required by Sagemaker for deployment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "#### This step Booster.save_model was needed before creating a tar.gz . \n",
    "# Otherwise I faced issues with prediction on deployment.\n",
    "\n",
    "mymodel._Booster.save_model(model_file_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DEMO-local-xgboost-model\n"
     ]
    }
   ],
   "source": [
    "!tar czvf model.tar.gz $model_file_name"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Upload the pre-trained model to S3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "sagemaker/DEMO-xgboost-byo/DEMO-local-xgboost-model/model.tar.gz\n"
     ]
    }
   ],
   "source": [
    "#### prefix in S3\n",
    "prefix = \"sagemaker/DEMO-xgboost-byo\"\n",
    "\n",
    "fObj = open(\"model.tar.gz\", \"rb\")\n",
    "key = os.path.join(prefix, model_file_name, \"model.tar.gz\")\n",
    "print(key)\n",
    "boto3.Session().resource(\"s3\").Bucket(bucket).Object(key).upload_fileobj(fObj)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Set up hosting for the model¶\n",
    "#### Import model into hosting\n",
    "This involves creating a SageMaker model from the model file previously uploaded to S3."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Create a Sagemaker model "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "The method get_image_uri has been renamed in sagemaker>=2.\n",
      "See: https://sagemaker.readthedocs.io/en/stable/v2.html for details.\n"
     ]
    }
   ],
   "source": [
    "from sagemaker.amazon.amazon_estimator import get_image_uri\n",
    "\n",
    "#### Get the built-in xgboost container image in Sagemaker to host our model\n",
    "#container = get_image_uri(boto3.Session().region_name, \"xgboost\", \"0.90-1\")\n",
    "container = get_image_uri(boto3.Session().region_name, \"xgboost\", \"1.7-1\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "https://s3-us-east-1.amazonaws.com/sagemaker-us-east-1-180494263813/sagemaker/DEMO-xgboost-byo/DEMO-local-xgboost-model/model.tar.gz\n",
      "arn:aws:sagemaker:us-east-1:180494263813:model/DEMO-local-xgboost-model2024-07-04-05-10-10\n",
      "CPU times: user 27.7 ms, sys: 1.34 ms, total: 29 ms\n",
      "Wall time: 1.15 s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "from time import gmtime, strftime\n",
    "\n",
    "model_name = model_file_name + strftime(\"%Y-%m-%d-%H-%M-%S\", gmtime())\n",
    "\n",
    "model_url = \"https://s3-{}.amazonaws.com/{}/{}\".format(region, bucket, key)\n",
    "\n",
    "sm_client = boto3.client(\"sagemaker\")\n",
    "\n",
    "print(model_url)\n",
    "\n",
    "primary_container = {\n",
    "    \"Image\": container,\n",
    "    \"ModelDataUrl\": model_url,\n",
    "}\n",
    "\n",
    "create_model_response2 = sm_client.create_model(\n",
    "    ModelName=model_name, ExecutionRoleArn=role, PrimaryContainer=primary_container\n",
    ")\n",
    "\n",
    "print(create_model_response2[\"ModelArn\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create endpoint configuration\n",
    "\n",
    "Create an endpoint configuration, that describes the distribution of traffic across the models, whether split, shadowed, or sampled in some way. In addition, the endpoint configuration describes the instance type required for model deployment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DEMO-XGBoostEndpointConfig-2024-07-04-05-10-11\n",
      "Endpoint Config Arn: arn:aws:sagemaker:us-east-1:180494263813:endpoint-config/DEMO-XGBoostEndpointConfig-2024-07-04-05-10-11\n"
     ]
    }
   ],
   "source": [
    "from time import gmtime, strftime\n",
    "\n",
    "endpoint_config_name = \"DEMO-XGBoostEndpointConfig-\" + strftime(\"%Y-%m-%d-%H-%M-%S\", gmtime())\n",
    "\n",
    "print(endpoint_config_name)\n",
    "\n",
    "create_endpoint_config_response = sm_client.create_endpoint_config(\n",
    "    EndpointConfigName=endpoint_config_name,\n",
    "    ProductionVariants=[\n",
    "        {\n",
    "            \"InstanceType\": \"ml.m4.xlarge\",\n",
    "            \"InitialInstanceCount\": 1,\n",
    "            \"InitialVariantWeight\": 1,\n",
    "            \"ModelName\": model_name,\n",
    "            \"VariantName\": \"AllTraffic\",\n",
    "        }\n",
    "    ],\n",
    ")\n",
    "\n",
    "print(\"Endpoint Config Arn: \" + create_endpoint_config_response[\"EndpointConfigArn\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Create endpoint\n",
    "Lastly, you create the endpoint that serves up the model, through specifying the name and configuration defined above. The end result is an endpoint that can be validated and incorporated into production applications. This takes 9-11 minutes to complete."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "DEMO-XGBoostEndpoint-2024-07-04-05-10-12\n",
      "arn:aws:sagemaker:us-east-1:180494263813:endpoint/DEMO-XGBoostEndpoint-2024-07-04-05-10-12\n",
      "Status: Creating\n",
      "Status: Creating\n",
      "Status: Creating\n",
      "Status: Creating\n",
      "Status: InService\n",
      "Arn: arn:aws:sagemaker:us-east-1:180494263813:endpoint/DEMO-XGBoostEndpoint-2024-07-04-05-10-12\n",
      "Status: InService\n",
      "CPU times: user 91.1 ms, sys: 5.15 ms, total: 96.3 ms\n",
      "Wall time: 4min 1s\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "import time\n",
    "\n",
    "endpoint_name = \"DEMO-XGBoostEndpoint-\" + strftime(\"%Y-%m-%d-%H-%M-%S\", gmtime())\n",
    "print(endpoint_name)\n",
    "create_endpoint_response = sm_client.create_endpoint(\n",
    "    EndpointName=endpoint_name, EndpointConfigName=endpoint_config_name\n",
    ")\n",
    "print(create_endpoint_response[\"EndpointArn\"])\n",
    "\n",
    "resp = sm_client.describe_endpoint(EndpointName=endpoint_name)\n",
    "status = resp[\"EndpointStatus\"]\n",
    "print(\"Status: \" + status)\n",
    "\n",
    "while status == \"Creating\":\n",
    "    time.sleep(60)\n",
    "    resp = sm_client.describe_endpoint(EndpointName=endpoint_name)\n",
    "    status = resp[\"EndpointStatus\"]\n",
    "    print(\"Status: \" + status)\n",
    "\n",
    "print(\"Arn: \" + resp[\"EndpointArn\"])\n",
    "print(\"Status: \" + status)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Validate the model for use\n",
    "Now you can obtain the endpoint from the client library using the result from previous operations and generate classifications from the model using that endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [],
   "source": [
    "runtime_client = boto3.client(\"runtime.sagemaker\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Lets generate the prediction. We'll pick csv data from the test data file"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Payload :\n",
      "\n",
      "5.400000000000000355e+00,3.000000000000000000e+00,4.500000000000000000e+00,1.500000000000000000e+00\n",
      "5.599999999999999645e+00,3.000000000000000000e+00,4.099999999999999645e+00,1.300000000000000044e+00\n",
      "6.299999999999999822e+00,2.799999999999999822e+00,5.099999999999999645e+00,1.500000000000000000e+00\n",
      "6.000000000000000000e+00,3.000000000000000000e+00,4.799999999999999822e+00,1.800000000000000044e+00\n",
      "5.099999999999999645e+00,3.299999999999999822e+00,1.699999999999999956e+00,5.000000000000000000e-01\n",
      "\n",
      "Results :\n",
      "\n",
      "\n",
      "\n",
      "Predicted Class Probabilities: 1.0\n",
      "1.0\n",
      "1.0\n",
      "2.0\n",
      "0.0\n",
      ".\n",
      "CPU times: user 23.3 ms, sys: 0 ns, total: 23.3 ms\n",
      "Wall time: 99 ms\n"
     ]
    }
   ],
   "source": [
    "%%time\n",
    "import json\n",
    "\n",
    "\n",
    "file_name = (\n",
    "    \"test_point.csv\"  # customize to your test file, will be 'mnist.single.test' if use data above\n",
    ")\n",
    "\n",
    "with open(file_name, \"r\") as f:\n",
    "    payload = f.read().strip()\n",
    "    \n",
    "    \n",
    "print(\"Payload :\\n\")\n",
    "\n",
    "print(payload)\n",
    "print()\n",
    "\n",
    "#payload = json.loads(payload)\n",
    "\n",
    "response = runtime_client.invoke_endpoint(\n",
    "    EndpointName=endpoint_name, ContentType=\"text/csv\", Body=payload\n",
    ")\n",
    "\n",
    "#print(response)\n",
    "\n",
    "print(\"Results :\\n\")\n",
    "print()\n",
    "#print(response[\"Body\"].read())\n",
    "result = response[\"Body\"].read().decode(\"ascii\")\n",
    "#result = response[\"Body\"].read().decode()\n",
    "# Unpack response\n",
    "print(\"\\nPredicted Class Probabilities: {}.\".format(result))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### (Optional) Delete the Endpoint\n",
    "\n",
    "If you're ready to be done with this notebook, please run the delete_endpoint line in the cell below.  This will remove the hosted endpoint you created and avoid any charges from a stray instance being left on."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'ResponseMetadata': {'RequestId': 'aaf9574a-a3c0-4f91-acb1-1cbb22ee839d',\n",
       "  'HTTPStatusCode': 200,\n",
       "  'HTTPHeaders': {'x-amzn-requestid': 'aaf9574a-a3c0-4f91-acb1-1cbb22ee839d',\n",
       "   'content-type': 'application/x-amz-json-1.1',\n",
       "   'date': 'Thu, 04 Jul 2024 05:19:15 GMT',\n",
       "   'content-length': '0'},\n",
       "  'RetryAttempts': 0}}"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "sm_client.delete_endpoint(EndpointName=endpoint_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.14"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}