05 - Train Models.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Train Models\n",
        "\n",
        "The central goal of machine learning is to train predictive models that can be used by applications. In Azure Machine Learning,  you can use scripts to train models leveraging common machine learning frameworks like Scikit-Learn, Tensorflow, PyTorch, SparkML, and others. You can run these training scripts as experiments in order to track metrics and outputs, which include the trained models."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Connect to your workspace\n",
        "\n",
        "To get started, connect to your workspace.\n",
        "\n",
        "> **Note**: If you haven't already established an authenticated session with your Azure subscription, you'll be prompted to authenticate by clicking a link, entering an authentication code, and signing into Azure."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "gather": {
          "logged": 1649366036303
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Ready to use Azure ML 1.44.0 to work with mllearningworkspace\n"
          ]
        }
      ],
      "source": [
        "import azureml.core\n",
        "from azureml.core import Workspace\n",
        "\n",
        "# Load the workspace from the saved config file\n",
        "ws = Workspace.from_config()\n",
        "print('Ready to use Azure ML {} to work with {}'.format(azureml.core.VERSION, ws.name))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Create a training script\n",
        "\n",
        "You're going to use a Python script to train a machine learning model based on the diabates data, so let's start by creating a folder for the script and data files."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "gather": {
          "logged": 1649366039039
        }
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "'diabetes-training/diabetes.csv'"
            ]
          },
          "execution_count": 2,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import os, shutil\n",
        "\n",
        "# Create a folder for the experiment files\n",
        "training_folder = 'diabetes-training'\n",
        "os.makedirs(training_folder, exist_ok=True)\n",
        "\n",
        "# Copy the data file into the experiment folder\n",
        "shutil.copy('data/diabetes.csv', os.path.join(training_folder, \"diabetes.csv\"))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now you're ready to create the training script and save it in the folder.\n",
        "\n",
        "> **Note**: This code *creates* the script - it doesn't run it!"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 3,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Writing diabetes-training/diabetes_training.py\n"
          ]
        }
      ],
      "source": [
        "%%writefile $training_folder/diabetes_training.py\n",
        "# Import libraries\n",
        "from azureml.core import Run\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import joblib\n",
        "import os\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.metrics import roc_auc_score\n",
        "from sklearn.metrics import roc_curve\n",
        "\n",
        "# Get the experiment run context\n",
        "run = Run.get_context()\n",
        "\n",
        "# load the diabetes dataset\n",
        "print(\"Loading Data...\")\n",
        "diabetes = pd.read_csv('diabetes.csv')\n",
        "\n",
        "# Separate features and labels\n",
        "X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values\n",
        "\n",
        "# Split data into training set and test set\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)\n",
        "\n",
        "# Set regularization hyperparameter\n",
        "reg = 0.01\n",
        "\n",
        "# Train a logistic regression model\n",
        "print('Training a logistic regression model with regularization rate of', reg)\n",
        "run.log('Regularization Rate',  np.float(reg))\n",
        "model = LogisticRegression(C=1/reg, solver=\"liblinear\").fit(X_train, y_train)\n",
        "\n",
        "# calculate accuracy\n",
        "y_hat = model.predict(X_test)\n",
        "acc = np.average(y_hat == y_test)\n",
        "print('Accuracy:', acc)\n",
        "run.log('Accuracy', np.float(acc))\n",
        "\n",
        "# calculate AUC\n",
        "y_scores = model.predict_proba(X_test)\n",
        "auc = roc_auc_score(y_test,y_scores[:,1])\n",
        "print('AUC: ' + str(auc))\n",
        "run.log('AUC', np.float(auc))\n",
        "\n",
        "# Save the trained model in the outputs folder\n",
        "os.makedirs('outputs', exist_ok=True)\n",
        "joblib.dump(value=model, filename='outputs/diabetes_model.pkl')\n",
        "\n",
        "run.complete()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Run the training script as an experiment\n",
        "\n",
        "Now you're ready to run the script as an experiment. Note that the default environment does not include the **scikit-learn** package, so you need to explicitly add that to the configuration. The conda environment is built on-demand the first time the experiment is run, and cached for future runs that use the same configuration; so the first run will take a little longer."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 4,
      "metadata": {
        "gather": {
          "logged": 1649366052967
        }
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "ca64316feb354f62ab1316df08322e1b",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "_UserRunWidget(widget_settings={'childWidgetDisplay': 'popup', 'send_telemetry': False, 'log_level': 'INFO', '…"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/aml.mini.widget.v1": "{\"status\": \"Completed\", \"workbench_run_details_uri\": \"https://ml.azure.com/runs/mslearn-train-diabetes_1670866976_b7d56abe?wsid=/subscriptions/376621af-6a06-4df6-b902-d25c6239f015/resourcegroups/learning/workspaces/mllearningworkspace&tid=16b3c013-d300-468d-ac64-7eda0820b6d3\", \"run_id\": \"mslearn-train-diabetes_1670866976_b7d56abe\", \"run_properties\": {\"run_id\": \"mslearn-train-diabetes_1670866976_b7d56abe\", \"created_utc\": \"2022-12-12T17:42:57.328879Z\", \"properties\": {\"_azureml.ComputeTargetType\": \"local\", \"ContentSnapshotId\": \"4ba9f894-1220-41a8-827a-d241840f87b5\"}, \"tags\": {\"mlflow.source.type\": \"JOB\", \"mlflow.source.name\": \"diabetes_training.py\"}, \"script_name\": null, \"arguments\": null, \"end_time_utc\": \"2022-12-12T17:48:43.85985Z\", \"status\": \"Completed\", \"log_files\": {\"azureml-logs/60_control_log.txt\": \"https://mllearningwork3345488027.blob.core.windows.net/azureml/ExperimentRun/dcid.mslearn-train-diabetes_1670866976_b7d56abe/azureml-logs/60_control_log.txt?sv=2019-07-07&sr=b&sig=Q%2Bl9JGOpg%2FFAuCv7JgCaR%2B7yhM4XA6rZUi2rxwCpmRg%3D&skoid=530a969d-1f1a-4d2a-9866-6f18d23e261b&sktid=16b3c013-d300-468d-ac64-7eda0820b6d3&skt=2022-12-12T16%3A43%3A14Z&ske=2022-12-14T00%3A53%3A14Z&sks=b&skv=2019-07-07&st=2022-12-12T17%3A43%3A49Z&se=2022-12-13T01%3A53%3A49Z&sp=r\", \"azureml-logs/70_driver_log.txt\": \"https://mllearningwork3345488027.blob.core.windows.net/azureml/ExperimentRun/dcid.mslearn-train-diabetes_1670866976_b7d56abe/azureml-logs/70_driver_log.txt?sv=2019-07-07&sr=b&sig=mpzJOQqrVLuaRvjztse%2B%2BprkDpbKKMeu%2F2g%2BZm7imLM%3D&skoid=530a969d-1f1a-4d2a-9866-6f18d23e261b&sktid=16b3c013-d300-468d-ac64-7eda0820b6d3&skt=2022-12-12T16%3A43%3A14Z&ske=2022-12-14T00%3A53%3A14Z&sks=b&skv=2019-07-07&st=2022-12-12T17%3A43%3A49Z&se=2022-12-13T01%3A53%3A49Z&sp=r\", \"logs/azureml/9_azureml.log\": \"https://mllearningwork3345488027.blob.core.windows.net/azureml/ExperimentRun/dcid.mslearn-train-diabetes_1670866976_b7d56abe/logs/azureml/9_azureml.log?sv=2019-07-07&sr=b&sig=CbkfYMUOHrUWw%2Fz53Yy3kIe8RNxLDbHJBZSxqpEdhtg%3D&skoid=530a969d-1f1a-4d2a-9866-6f18d23e261b&sktid=16b3c013-d300-468d-ac64-7eda0820b6d3&skt=2022-12-12T16%3A43%3A14Z&ske=2022-12-14T00%3A53%3A14Z&sks=b&skv=2019-07-07&st=2022-12-12T17%3A43%3A49Z&se=2022-12-13T01%3A53%3A49Z&sp=r\"}, \"log_groups\": [[\"logs/azureml/9_azureml.log\"], [\"azureml-logs/60_control_log.txt\"], [\"azureml-logs/70_driver_log.txt\"]], \"run_duration\": \"0:05:46\", \"run_number\": \"1670866977\", \"run_queued_details\": {\"status\": \"Completed\", \"details\": null}}, \"child_runs\": [], \"children_metrics\": {}, \"run_metrics\": [{\"name\": \"Regularization Rate\", \"run_id\": \"mslearn-train-diabetes_1670866976_b7d56abe\", \"categories\": [0], \"series\": [{\"data\": [0.01]}]}, {\"name\": \"Accuracy\", \"run_id\": \"mslearn-train-diabetes_1670866976_b7d56abe\", \"categories\": [0], \"series\": [{\"data\": [0.774]}]}, {\"name\": \"AUC\", \"run_id\": \"mslearn-train-diabetes_1670866976_b7d56abe\", \"categories\": [0], \"series\": [{\"data\": [0.8484929598487486]}]}], \"run_logs\": \"[2022-12-12T17:48:28.025175] Entering context manager injector.\\nCannot provide tracer without any exporter configured.\\n/azureml-envs/azureml_809a074975457de1dd27bdfcf2d79d61/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.\\n  from cryptography.hazmat.backends import default_backend\\n[2022-12-12T17:48:29.687717] context_manager_injector.py Command line Options: Namespace(inject=['ProjectPythonPath:context_managers.ProjectPythonPath', 'RunHistory:context_managers.RunHistory', 'TrackUserError:context_managers.TrackUserError'], invocation=['diabetes_training.py'])\\nScript type = None\\n[2022-12-12T17:48:29.693495] Entering Run History Context Manager.\\n/azureml-envs/azureml_809a074975457de1dd27bdfcf2d79d61/lib/python3.6/site-packages/azureml/history/_tracking.py:367: FutureWarning: azureml.core: AzureML support for Python 3.6 is deprecated and will be dropped in an upcoming release. At that point, existing Python 3.6 workflows that use AzureML will continue to work without modification, but Python 3.6 users will no longer get access to the latest AzureML features and bugfixes. We recommend that you upgrade to Python 3.7 or newer. To disable SDK V1 deprecation warning set the environment variable AZUREML_DEPRECATE_WARNING to 'False'\\n  from azureml.core.run import Run\\n/azureml-envs/azureml_809a074975457de1dd27bdfcf2d79d61/lib/python3.6/site-packages/azureml/history/_tracking.py:186: FutureWarning: MLflow support for Python 3.6 is deprecated and will be dropped in an upcoming release. At that point, existing Python 3.6 workflows that use MLflow will continue to work without modification, but Python 3.6 users will no longer get access to the latest MLflow features and bugfixes. We recommend that you upgrade to Python 3.7 or newer.\\n  import mlflow\\n[2022-12-12T17:48:33.785612] Current directory: /azureml-run\\n[2022-12-12T17:48:33.785735] Preparing to call script [diabetes_training.py] with arguments:[]\\n[2022-12-12T17:48:33.785764] After variable expansion, calling script [diabetes_training.py] with arguments:[]\\n\\nLoading Data...\\nTraining a logistic regression model with regularization rate of 0.01\\nAccuracy: 0.774\\nAUC: 0.8484929598487486\\n\\n\\n[2022-12-12T17:48:40.708224] The experiment completed successfully. Finalizing run...\\n[2022-12-12T17:48:40.708251] Start FinalizingInRunHistory\\n[2022-12-12T17:48:40.711164] Logging experiment finalizing status in history service.\\nStarting the daemon thread to refresh tokens in background for process with pid = 9\\nCleaning up all outstanding Run operations, waiting 300.0 seconds\\n2 items cleaning up...\\nCleanup took 0.14078712463378906 seconds\\n[2022-12-12T17:48:41.852462] Finished context manager injector.\\n\\nRun is completed.\", \"graph\": {}, \"widget_settings\": {\"childWidgetDisplay\": \"popup\", \"send_telemetry\": false, \"log_level\": \"INFO\", \"sdk_version\": \"1.44.0\"}, \"loading\": false}"
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/plain": [
              "{'runId': 'mslearn-train-diabetes_1670866976_b7d56abe',\n",
              " 'target': 'local',\n",
              " 'status': 'Completed',\n",
              " 'startTimeUtc': '2022-12-12T17:48:26.586939Z',\n",
              " 'endTimeUtc': '2022-12-12T17:48:43.85985Z',\n",
              " 'services': {},\n",
              " 'properties': {'_azureml.ComputeTargetType': 'local',\n",
              "  'ContentSnapshotId': '4ba9f894-1220-41a8-827a-d241840f87b5'},\n",
              " 'inputDatasets': [],\n",
              " 'outputDatasets': [],\n",
              " 'runDefinition': {'script': 'diabetes_training.py',\n",
              "  'command': '',\n",
              "  'useAbsolutePath': False,\n",
              "  'arguments': [],\n",
              "  'sourceDirectoryDataStore': None,\n",
              "  'framework': 'Python',\n",
              "  'communicator': 'None',\n",
              "  'target': 'local',\n",
              "  'dataReferences': {},\n",
              "  'data': {},\n",
              "  'outputData': {},\n",
              "  'datacaches': [],\n",
              "  'jobName': None,\n",
              "  'maxRunDurationSeconds': 2592000,\n",
              "  'nodeCount': 1,\n",
              "  'instanceTypes': [],\n",
              "  'priority': None,\n",
              "  'credentialPassthrough': False,\n",
              "  'identity': None,\n",
              "  'environment': {'name': 'experiment_env',\n",
              "   'version': 'Autosave_2022-12-12T17:42:57Z_f8a26027',\n",
              "   'assetId': 'azureml://locations/westeurope/workspaces/8c877630-3281-4ea1-89cf-f8f8040c7370/environments/experiment_env/versions/Autosave_2022-12-12T17:42:57Z_f8a26027',\n",
              "   'autoRebuild': True,\n",
              "   'python': {'interpreterPath': 'python',\n",
              "    'userManagedDependencies': False,\n",
              "    'condaDependencies': {'dependencies': ['python=3.6.2',\n",
              "      'scikit-learn',\n",
              "      'pandas',\n",
              "      'pip',\n",
              "      {'pip': ['azureml-defaults', 'azureml-mlflow']}],\n",
              "     'name': 'simple_environment'},\n",
              "    'baseCondaEnvironment': None},\n",
              "   'environmentVariables': {'EXAMPLE_ENV_VAR': 'EXAMPLE_VALUE'},\n",
              "   'docker': {'baseImage': 'mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:20220708.v1',\n",
              "    'platform': {'os': 'Linux', 'architecture': 'amd64'},\n",
              "    'baseDockerfile': None,\n",
              "    'baseImageRegistry': {'address': None, 'username': None, 'password': None},\n",
              "    'enabled': False,\n",
              "    'arguments': []},\n",
              "   'spark': {'repositories': [], 'packages': [], 'precachePackages': True},\n",
              "   'inferencingStackVersion': None},\n",
              "  'history': {'outputCollection': True,\n",
              "   'directoriesToWatch': ['logs'],\n",
              "   'enableMLflowTracking': True,\n",
              "   'snapshotProject': True},\n",
              "  'spark': {'configuration': {'spark.app.name': 'Azure ML Experiment',\n",
              "    'spark.yarn.maxAppAttempts': '1'}},\n",
              "  'parallelTask': {'maxRetriesPerWorker': 0,\n",
              "   'workerCountPerNode': 1,\n",
              "   'terminalExitCodes': None,\n",
              "   'configuration': {}},\n",
              "  'amlCompute': {'name': None,\n",
              "   'vmSize': None,\n",
              "   'retainCluster': False,\n",
              "   'clusterMaxNodeCount': None},\n",
              "  'aiSuperComputer': {'instanceType': 'D2',\n",
              "   'imageVersion': None,\n",
              "   'location': None,\n",
              "   'aiSuperComputerStorageData': None,\n",
              "   'interactive': False,\n",
              "   'scalePolicy': None,\n",
              "   'virtualClusterArmId': None,\n",
              "   'tensorboardLogDirectory': None,\n",
              "   'sshPublicKey': None,\n",
              "   'sshPublicKeys': None,\n",
              "   'enableAzmlInt': True,\n",
              "   'priority': 'Medium',\n",
              "   'slaTier': 'Standard',\n",
              "   'userAlias': None},\n",
              "  'kubernetesCompute': {'instanceType': None},\n",
              "  'tensorflow': {'workerCount': 1, 'parameterServerCount': 1},\n",
              "  'mpi': {'processCountPerNode': 1},\n",
              "  'pyTorch': {'communicationBackend': 'nccl', 'processCount': None},\n",
              "  'hdi': {'yarnDeployMode': 'Cluster'},\n",
              "  'containerInstance': {'region': None, 'cpuCores': 2.0, 'memoryGb': 3.5},\n",
              "  'exposedPorts': None,\n",
              "  'docker': {'useDocker': True,\n",
              "   'sharedVolumes': True,\n",
              "   'shmSize': '2g',\n",
              "   'arguments': []},\n",
              "  'cmk8sCompute': {'configuration': {}},\n",
              "  'commandReturnCodeConfig': {'returnCode': 'Zero',\n",
              "   'successfulReturnCodes': []},\n",
              "  'environmentVariables': {},\n",
              "  'applicationEndpoints': {},\n",
              "  'parameters': []},\n",
              " 'logFiles': {'azureml-logs/60_control_log.txt': 'https://mllearningwork3345488027.blob.core.windows.net/azureml/ExperimentRun/dcid.mslearn-train-diabetes_1670866976_b7d56abe/azureml-logs/60_control_log.txt?sv=2019-07-07&sr=b&sig=3syOGzvIVOQ83a6BND5h%2FQAKULyg6J%2Fbe9FuvnvkL6w%3D&skoid=530a969d-1f1a-4d2a-9866-6f18d23e261b&sktid=16b3c013-d300-468d-ac64-7eda0820b6d3&skt=2022-12-12T16%3A43%3A14Z&ske=2022-12-14T00%3A53%3A14Z&sks=b&skv=2019-07-07&st=2022-12-12T17%3A38%3A48Z&se=2022-12-13T01%3A48%3A48Z&sp=r',\n",
              "  'azureml-logs/70_driver_log.txt': 'https://mllearningwork3345488027.blob.core.windows.net/azureml/ExperimentRun/dcid.mslearn-train-diabetes_1670866976_b7d56abe/azureml-logs/70_driver_log.txt?sv=2019-07-07&sr=b&sig=mKtTkrC%2BQ4IPw2tewfndD%2F7v5624nXWPRpWLDTLjXbo%3D&skoid=530a969d-1f1a-4d2a-9866-6f18d23e261b&sktid=16b3c013-d300-468d-ac64-7eda0820b6d3&skt=2022-12-12T16%3A43%3A14Z&ske=2022-12-14T00%3A53%3A14Z&sks=b&skv=2019-07-07&st=2022-12-12T17%3A38%3A48Z&se=2022-12-13T01%3A48%3A48Z&sp=r',\n",
              "  'logs/azureml/9_azureml.log': 'https://mllearningwork3345488027.blob.core.windows.net/azureml/ExperimentRun/dcid.mslearn-train-diabetes_1670866976_b7d56abe/logs/azureml/9_azureml.log?sv=2019-07-07&sr=b&sig=9XirROf3xv%2BC%2F0YPSUnGRpM0scphoyLAuiBP8HAycvE%3D&skoid=530a969d-1f1a-4d2a-9866-6f18d23e261b&sktid=16b3c013-d300-468d-ac64-7eda0820b6d3&skt=2022-12-12T16%3A43%3A14Z&ske=2022-12-14T00%3A53%3A14Z&sks=b&skv=2019-07-07&st=2022-12-12T17%3A38%3A40Z&se=2022-12-13T01%3A48%3A40Z&sp=r'},\n",
              " 'submittedBy': 'Carlo Binda'}"
            ]
          },
          "execution_count": 4,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "from azureml.core import Experiment, ScriptRunConfig, Environment\n",
        "from azureml.core.runconfig import DockerConfiguration\n",
        "from azureml.widgets import RunDetails\n",
        "\n",
        "# Create a Python environment for the experiment (from a .yml file)\n",
        "env = Environment.from_conda_specification(\"experiment_env\", \"environment.yml\")\n",
        "\n",
        "# Create a script config\n",
        "script_config = ScriptRunConfig(source_directory=training_folder,\n",
        "                                script='diabetes_training.py',\n",
        "                                environment=env,\n",
        "                                docker_runtime_config=DockerConfiguration(use_docker=True)) \n",
        "\n",
        "# submit the experiment run\n",
        "experiment_name = 'mslearn-train-diabetes'\n",
        "experiment = Experiment(workspace=ws, name=experiment_name)\n",
        "run = experiment.submit(config=script_config)\n",
        "\n",
        "# Show the running experiment run in the notebook widget\n",
        "RunDetails(run).show()\n",
        "\n",
        "# Block until the experiment run has completed\n",
        "run.wait_for_completion()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "You can retrieve the metrics and outputs from the **Run** object."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 5,
      "metadata": {
        "gather": {
          "logged": 1649366055677
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Regularization Rate 0.01\n",
            "Accuracy 0.774\n",
            "AUC 0.8484929598487486\n",
            "\n",
            "\n",
            "azureml-logs/60_control_log.txt\n",
            "azureml-logs/70_driver_log.txt\n",
            "logs/azureml/9_azureml.log\n",
            "outputs/diabetes_model.pkl\n"
          ]
        }
      ],
      "source": [
        "# Get logged metrics and files\n",
        "metrics = run.get_metrics()\n",
        "for key in metrics.keys():\n",
        "        print(key, metrics.get(key))\n",
        "print('\\n')\n",
        "for file in run.get_file_names():\n",
        "    print(file)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Register the trained model\n",
        "\n",
        "Note that the outputs of the experiment include the trained model file (**diabetes_model.pkl**). You can register this model in your Azure Machine Learning workspace, making it possible to track model versions and retrieve them later."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 6,
      "metadata": {
        "gather": {
          "logged": 1649366061163
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "diabetes_model version: 1\n",
            "\t Training context : Script\n",
            "\t AUC : 0.8484929598487486\n",
            "\t Accuracy : 0.774\n",
            "\n",
            "\n"
          ]
        }
      ],
      "source": [
        "from azureml.core import Model\n",
        "\n",
        "# Register the model\n",
        "run.register_model(model_path='outputs/diabetes_model.pkl', model_name='diabetes_model',\n",
        "                   tags={'Training context':'Script'},\n",
        "                   properties={'AUC': run.get_metrics()['AUC'], 'Accuracy': run.get_metrics()['Accuracy']})\n",
        "\n",
        "# List registered models\n",
        "for model in Model.list(ws):\n",
        "    print(model.name, 'version:', model.version)\n",
        "    for tag_name in model.tags:\n",
        "        tag = model.tags[tag_name]\n",
        "        print ('\\t',tag_name, ':', tag)\n",
        "    for prop_name in model.properties:\n",
        "        prop = model.properties[prop_name]\n",
        "        print ('\\t',prop_name, ':', prop)\n",
        "    print('\\n')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "> **Note**: The code above uses *properties* and *tags* to note additional information about the model. The key difference between properties and tags is that properties cannot be changed after the model is registered, while tags for a registered model can be modified.\n",
        "\n",
        "## Create a parameterized training script\n",
        "\n",
        "You can increase the flexibility of your training experiment by adding parameters to your script, enabling you to repeat the same training experiment with different settings. In this case, you'll add a parameter for the regularization rate used by the logistic regression algorithm when training the model.\n",
        "\n",
        "Again, lets start by creating a folder for the parameterized script and the training data."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 7,
      "metadata": {
        "gather": {
          "logged": 1649366064098
        }
      },
      "outputs": [
        {
          "data": {
            "text/plain": [
              "'diabetes-training-params/diabetes.csv'"
            ]
          },
          "execution_count": 7,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "import os, shutil\n",
        "\n",
        "# Create a folder for the experiment files\n",
        "training_folder = 'diabetes-training-params'\n",
        "os.makedirs(training_folder, exist_ok=True)\n",
        "\n",
        "# Copy the data file into the experiment folder\n",
        "shutil.copy('data/diabetes.csv', os.path.join(training_folder, \"diabetes.csv\"))"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Now let's create a script with an argument for the regularization rate hyperparameter. The argument is read using a Python **argparse.ArgumentParser** object."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 8,
      "metadata": {},
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Writing diabetes-training-params/diabetes_training.py\n"
          ]
        }
      ],
      "source": [
        "%%writefile $training_folder/diabetes_training.py\n",
        "# Import libraries\n",
        "from azureml.core import Run\n",
        "import pandas as pd\n",
        "import numpy as np\n",
        "import joblib\n",
        "import os\n",
        "import argparse\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.linear_model import LogisticRegression\n",
        "from sklearn.metrics import roc_auc_score\n",
        "from sklearn.metrics import roc_curve\n",
        "\n",
        "# Get the experiment run context\n",
        "run = Run.get_context()\n",
        "\n",
        "# Set regularization hyperparameter\n",
        "parser = argparse.ArgumentParser()\n",
        "parser.add_argument('--reg_rate', type=float, dest='reg', default=0.01)\n",
        "args = parser.parse_args()\n",
        "reg = args.reg\n",
        "\n",
        "# load the diabetes dataset\n",
        "print(\"Loading Data...\")\n",
        "# load the diabetes dataset\n",
        "diabetes = pd.read_csv('diabetes.csv')\n",
        "\n",
        "# Separate features and labels\n",
        "X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values\n",
        "\n",
        "# Split data into training set and test set\n",
        "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)\n",
        "\n",
        "# Train a logistic regression model\n",
        "print('Training a logistic regression model with regularization rate of', reg)\n",
        "run.log('Regularization Rate',  np.float(reg))\n",
        "model = LogisticRegression(C=1/reg, solver=\"liblinear\").fit(X_train, y_train)\n",
        "\n",
        "# calculate accuracy\n",
        "y_hat = model.predict(X_test)\n",
        "acc = np.average(y_hat == y_test)\n",
        "print('Accuracy:', acc)\n",
        "run.log('Accuracy', np.float(acc))\n",
        "\n",
        "# calculate AUC\n",
        "y_scores = model.predict_proba(X_test)\n",
        "auc = roc_auc_score(y_test,y_scores[:,1])\n",
        "print('AUC: ' + str(auc))\n",
        "run.log('AUC', np.float(auc))\n",
        "\n",
        "os.makedirs('outputs', exist_ok=True)\n",
        "joblib.dump(value=model, filename='outputs/diabetes_model.pkl')\n",
        "\n",
        "run.complete()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Run the script with arguments\n",
        "\n",
        "You run the script as an experiment like you did previously, reusing the environment you created; but this time you must provide the **--reg_rate** parameter that the script expects as an argument."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 9,
      "metadata": {
        "gather": {
          "logged": 1649366078595
        }
      },
      "outputs": [
        {
          "data": {
            "application/vnd.jupyter.widget-view+json": {
              "model_id": "dbc4187004d44a3b8920a6273683493a",
              "version_major": 2,
              "version_minor": 0
            },
            "text/plain": [
              "_UserRunWidget(widget_settings={'childWidgetDisplay': 'popup', 'send_telemetry': False, 'log_level': 'INFO', '…"
            ]
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "application/aml.mini.widget.v1": "{\"status\": \"Completed\", \"workbench_run_details_uri\": \"https://ml.azure.com/runs/mslearn-train-diabetes_1670867622_0cfc8132?wsid=/subscriptions/376621af-6a06-4df6-b902-d25c6239f015/resourcegroups/learning/workspaces/mllearningworkspace&tid=16b3c013-d300-468d-ac64-7eda0820b6d3\", \"run_id\": \"mslearn-train-diabetes_1670867622_0cfc8132\", \"run_properties\": {\"run_id\": \"mslearn-train-diabetes_1670867622_0cfc8132\", \"created_utc\": \"2022-12-12T17:53:43.518684Z\", \"properties\": {\"_azureml.ComputeTargetType\": \"local\", \"ContentSnapshotId\": \"ea9c1836-5b35-4ab0-a1a0-79d2f1926404\"}, \"tags\": {\"mlflow.source.type\": \"JOB\", \"mlflow.source.name\": \"diabetes_training.py\"}, \"script_name\": null, \"arguments\": null, \"end_time_utc\": \"2022-12-12T17:54:03.435354Z\", \"status\": \"Completed\", \"log_files\": {\"azureml-logs/60_control_log.txt\": \"https://mllearningwork3345488027.blob.core.windows.net/azureml/ExperimentRun/dcid.mslearn-train-diabetes_1670867622_0cfc8132/azureml-logs/60_control_log.txt?sv=2019-07-07&sr=b&sig=ouESf923ofp%2BeCIS0mTCr4xZB5mTANO7Y6m80Dc%2FGUg%3D&skoid=530a969d-1f1a-4d2a-9866-6f18d23e261b&sktid=16b3c013-d300-468d-ac64-7eda0820b6d3&skt=2022-12-12T16%3A43%3A14Z&ske=2022-12-14T00%3A53%3A14Z&sks=b&skv=2019-07-07&st=2022-12-12T17%3A44%3A09Z&se=2022-12-13T01%3A54%3A09Z&sp=r\", \"azureml-logs/70_driver_log.txt\": \"https://mllearningwork3345488027.blob.core.windows.net/azureml/ExperimentRun/dcid.mslearn-train-diabetes_1670867622_0cfc8132/azureml-logs/70_driver_log.txt?sv=2019-07-07&sr=b&sig=28bC5vwdsN%2BtZ3CHgN3p%2BsM%2BfVLbfJMAF8%2Bhhf8FA60%3D&skoid=530a969d-1f1a-4d2a-9866-6f18d23e261b&sktid=16b3c013-d300-468d-ac64-7eda0820b6d3&skt=2022-12-12T16%3A43%3A14Z&ske=2022-12-14T00%3A53%3A14Z&sks=b&skv=2019-07-07&st=2022-12-12T17%3A44%3A09Z&se=2022-12-13T01%3A54%3A09Z&sp=r\", \"logs/azureml/8_azureml.log\": \"https://mllearningwork3345488027.blob.core.windows.net/azureml/ExperimentRun/dcid.mslearn-train-diabetes_1670867622_0cfc8132/logs/azureml/8_azureml.log?sv=2019-07-07&sr=b&sig=48q5FfNNcyhOM9fobYHFJpvA%2FbnWOHKFk3%2FLjPfBC2M%3D&skoid=530a969d-1f1a-4d2a-9866-6f18d23e261b&sktid=16b3c013-d300-468d-ac64-7eda0820b6d3&skt=2022-12-12T16%3A43%3A14Z&ske=2022-12-14T00%3A53%3A14Z&sks=b&skv=2019-07-07&st=2022-12-12T17%3A43%3A57Z&se=2022-12-13T01%3A53%3A57Z&sp=r\"}, \"log_groups\": [[\"logs/azureml/8_azureml.log\"], [\"azureml-logs/60_control_log.txt\"], [\"azureml-logs/70_driver_log.txt\"]], \"run_duration\": \"0:00:19\", \"run_number\": \"1670867623\", \"run_queued_details\": {\"status\": \"Completed\", \"details\": null}}, \"child_runs\": [], \"children_metrics\": {}, \"run_metrics\": [{\"name\": \"Regularization Rate\", \"run_id\": \"mslearn-train-diabetes_1670867622_0cfc8132\", \"categories\": [0], \"series\": [{\"data\": [0.1]}]}, {\"name\": \"Accuracy\", \"run_id\": \"mslearn-train-diabetes_1670867622_0cfc8132\", \"categories\": [0], \"series\": [{\"data\": [0.774]}]}, {\"name\": \"AUC\", \"run_id\": \"mslearn-train-diabetes_1670867622_0cfc8132\", \"categories\": [0], \"series\": [{\"data\": [0.8483198169063138]}]}], \"run_logs\": \"[2022-12-12T17:53:47.267224] Entering context manager injector.\\nCannot provide tracer without any exporter configured.\\n/azureml-envs/azureml_809a074975457de1dd27bdfcf2d79d61/lib/python3.6/site-packages/paramiko/transport.py:33: CryptographyDeprecationWarning: Python 3.6 is no longer supported by the Python core team. Therefore, support for it is deprecated in cryptography and will be removed in a future release.\\n  from cryptography.hazmat.backends import default_backend\\n[2022-12-12T17:53:48.869974] context_manager_injector.py Command line Options: Namespace(inject=['ProjectPythonPath:context_managers.ProjectPythonPath', 'RunHistory:context_managers.RunHistory', 'TrackUserError:context_managers.TrackUserError'], invocation=['diabetes_training.py', '--reg_rate', '0.1'])\\nScript type = None\\n[2022-12-12T17:53:48.878131] Entering Run History Context Manager.\\n/azureml-envs/azureml_809a074975457de1dd27bdfcf2d79d61/lib/python3.6/site-packages/azureml/history/_tracking.py:367: FutureWarning: azureml.core: AzureML support for Python 3.6 is deprecated and will be dropped in an upcoming release. At that point, existing Python 3.6 workflows that use AzureML will continue to work without modification, but Python 3.6 users will no longer get access to the latest AzureML features and bugfixes. We recommend that you upgrade to Python 3.7 or newer. To disable SDK V1 deprecation warning set the environment variable AZUREML_DEPRECATE_WARNING to 'False'\\n  from azureml.core.run import Run\\n/azureml-envs/azureml_809a074975457de1dd27bdfcf2d79d61/lib/python3.6/site-packages/azureml/history/_tracking.py:186: FutureWarning: MLflow support for Python 3.6 is deprecated and will be dropped in an upcoming release. At that point, existing Python 3.6 workflows that use MLflow will continue to work without modification, but Python 3.6 users will no longer get access to the latest MLflow features and bugfixes. We recommend that you upgrade to Python 3.7 or newer.\\n  import mlflow\\n[2022-12-12T17:53:53.223835] Current directory: /azureml-run\\n[2022-12-12T17:53:53.223998] Preparing to call script [diabetes_training.py] with arguments:['--reg_rate', '0.1']\\n[2022-12-12T17:53:53.224038] After variable expansion, calling script [diabetes_training.py] with arguments:['--reg_rate', '0.1']\\n\\nLoading Data...\\nTraining a logistic regression model with regularization rate of 0.1\\nAccuracy: 0.774\\nAUC: 0.8483198169063138\\n\\n\\n[2022-12-12T17:54:00.177856] The experiment completed successfully. Finalizing run...\\n[2022-12-12T17:54:00.177882] Start FinalizingInRunHistory\\n[2022-12-12T17:54:00.182605] Logging experiment finalizing status in history service.\\nStarting the daemon thread to refresh tokens in background for process with pid = 8\\nCleaning up all outstanding Run operations, waiting 300.0 seconds\\n2 items cleaning up...\\nCleanup took 0.14310097694396973 seconds\\n[2022-12-12T17:54:01.313368] Finished context manager injector.\\n\\nRun is completed.\", \"graph\": {}, \"widget_settings\": {\"childWidgetDisplay\": \"popup\", \"send_telemetry\": false, \"log_level\": \"INFO\", \"sdk_version\": \"1.44.0\"}, \"loading\": false}"
          },
          "metadata": {},
          "output_type": "display_data"
        },
        {
          "data": {
            "text/plain": [
              "{'runId': 'mslearn-train-diabetes_1670867622_0cfc8132',\n",
              " 'target': 'local',\n",
              " 'status': 'Finalizing',\n",
              " 'startTimeUtc': '2022-12-12T17:53:45.697696Z',\n",
              " 'services': {},\n",
              " 'properties': {'_azureml.ComputeTargetType': 'local',\n",
              "  'ContentSnapshotId': 'ea9c1836-5b35-4ab0-a1a0-79d2f1926404'},\n",
              " 'inputDatasets': [],\n",
              " 'outputDatasets': [],\n",
              " 'runDefinition': {'script': 'diabetes_training.py',\n",
              "  'command': '',\n",
              "  'useAbsolutePath': False,\n",
              "  'arguments': ['--reg_rate', '0.1'],\n",
              "  'sourceDirectoryDataStore': None,\n",
              "  'framework': 'Python',\n",
              "  'communicator': 'None',\n",
              "  'target': 'local',\n",
              "  'dataReferences': {},\n",
              "  'data': {},\n",
              "  'outputData': {},\n",
              "  'datacaches': [],\n",
              "  'jobName': None,\n",
              "  'maxRunDurationSeconds': 2592000,\n",
              "  'nodeCount': 1,\n",
              "  'instanceTypes': [],\n",
              "  'priority': None,\n",
              "  'credentialPassthrough': False,\n",
              "  'identity': None,\n",
              "  'environment': {'name': 'experiment_env',\n",
              "   'version': 'Autosave_2022-12-12T17:42:57Z_f8a26027',\n",
              "   'assetId': 'azureml://locations/westeurope/workspaces/8c877630-3281-4ea1-89cf-f8f8040c7370/environments/experiment_env/versions/Autosave_2022-12-12T17:42:57Z_f8a26027',\n",
              "   'autoRebuild': True,\n",
              "   'python': {'interpreterPath': 'python',\n",
              "    'userManagedDependencies': False,\n",
              "    'condaDependencies': {'dependencies': ['python=3.6.2',\n",
              "      'scikit-learn',\n",
              "      'pandas',\n",
              "      'pip',\n",
              "      {'pip': ['azureml-defaults', 'azureml-mlflow']}],\n",
              "     'name': 'simple_environment'},\n",
              "    'baseCondaEnvironment': None},\n",
              "   'environmentVariables': {'EXAMPLE_ENV_VAR': 'EXAMPLE_VALUE'},\n",
              "   'docker': {'baseImage': 'mcr.microsoft.com/azureml/openmpi4.1.0-ubuntu20.04:20220708.v1',\n",
              "    'platform': {'os': 'Linux', 'architecture': 'amd64'},\n",
              "    'baseDockerfile': None,\n",
              "    'baseImageRegistry': {'address': None, 'username': None, 'password': None},\n",
              "    'enabled': False,\n",
              "    'arguments': []},\n",
              "   'spark': {'repositories': [], 'packages': [], 'precachePackages': True},\n",
              "   'inferencingStackVersion': None},\n",
              "  'history': {'outputCollection': True,\n",
              "   'directoriesToWatch': ['logs'],\n",
              "   'enableMLflowTracking': True,\n",
              "   'snapshotProject': True},\n",
              "  'spark': {'configuration': {'spark.app.name': 'Azure ML Experiment',\n",
              "    'spark.yarn.maxAppAttempts': '1'}},\n",
              "  'parallelTask': {'maxRetriesPerWorker': 0,\n",
              "   'workerCountPerNode': 1,\n",
              "   'terminalExitCodes': None,\n",
              "   'configuration': {}},\n",
              "  'amlCompute': {'name': None,\n",
              "   'vmSize': None,\n",
              "   'retainCluster': False,\n",
              "   'clusterMaxNodeCount': None},\n",
              "  'aiSuperComputer': {'instanceType': 'D2',\n",
              "   'imageVersion': None,\n",
              "   'location': None,\n",
              "   'aiSuperComputerStorageData': None,\n",
              "   'interactive': False,\n",
              "   'scalePolicy': None,\n",
              "   'virtualClusterArmId': None,\n",
              "   'tensorboardLogDirectory': None,\n",
              "   'sshPublicKey': None,\n",
              "   'sshPublicKeys': None,\n",
              "   'enableAzmlInt': True,\n",
              "   'priority': 'Medium',\n",
              "   'slaTier': 'Standard',\n",
              "   'userAlias': None},\n",
              "  'kubernetesCompute': {'instanceType': None},\n",
              "  'tensorflow': {'workerCount': 1, 'parameterServerCount': 1},\n",
              "  'mpi': {'processCountPerNode': 1},\n",
              "  'pyTorch': {'communicationBackend': 'nccl', 'processCount': None},\n",
              "  'hdi': {'yarnDeployMode': 'Cluster'},\n",
              "  'containerInstance': {'region': None, 'cpuCores': 2.0, 'memoryGb': 3.5},\n",
              "  'exposedPorts': None,\n",
              "  'docker': {'useDocker': True,\n",
              "   'sharedVolumes': True,\n",
              "   'shmSize': '2g',\n",
              "   'arguments': []},\n",
              "  'cmk8sCompute': {'configuration': {}},\n",
              "  'commandReturnCodeConfig': {'returnCode': 'Zero',\n",
              "   'successfulReturnCodes': []},\n",
              "  'environmentVariables': {},\n",
              "  'applicationEndpoints': {},\n",
              "  'parameters': []},\n",
              " 'logFiles': {'azureml-logs/60_control_log.txt': 'https://mllearningwork3345488027.blob.core.windows.net/azureml/ExperimentRun/dcid.mslearn-train-diabetes_1670867622_0cfc8132/azureml-logs/60_control_log.txt?sv=2019-07-07&sr=b&sig=wIltSk8TcXTb5wz28WZfNb%2BAjn2IgyNueokcBU3%2BDtU%3D&skoid=530a969d-1f1a-4d2a-9866-6f18d23e261b&sktid=16b3c013-d300-468d-ac64-7eda0820b6d3&skt=2022-12-12T16%3A43%3A14Z&ske=2022-12-14T00%3A53%3A14Z&sks=b&skv=2019-07-07&st=2022-12-12T17%3A43%3A57Z&se=2022-12-13T01%3A53%3A57Z&sp=r',\n",
              "  'azureml-logs/70_driver_log.txt': 'https://mllearningwork3345488027.blob.core.windows.net/azureml/ExperimentRun/dcid.mslearn-train-diabetes_1670867622_0cfc8132/azureml-logs/70_driver_log.txt?sv=2019-07-07&sr=b&sig=4mXcO9cQNQEM8ztxWP7EJ0%2Fh6Dhp9HQPonjXSkpyAzg%3D&skoid=530a969d-1f1a-4d2a-9866-6f18d23e261b&sktid=16b3c013-d300-468d-ac64-7eda0820b6d3&skt=2022-12-12T16%3A43%3A14Z&ske=2022-12-14T00%3A53%3A14Z&sks=b&skv=2019-07-07&st=2022-12-12T17%3A43%3A57Z&se=2022-12-13T01%3A53%3A57Z&sp=r',\n",
              "  'logs/azureml/8_azureml.log': 'https://mllearningwork3345488027.blob.core.windows.net/azureml/ExperimentRun/dcid.mslearn-train-diabetes_1670867622_0cfc8132/logs/azureml/8_azureml.log?sv=2019-07-07&sr=b&sig=48q5FfNNcyhOM9fobYHFJpvA%2FbnWOHKFk3%2FLjPfBC2M%3D&skoid=530a969d-1f1a-4d2a-9866-6f18d23e261b&sktid=16b3c013-d300-468d-ac64-7eda0820b6d3&skt=2022-12-12T16%3A43%3A14Z&ske=2022-12-14T00%3A53%3A14Z&sks=b&skv=2019-07-07&st=2022-12-12T17%3A43%3A57Z&se=2022-12-13T01%3A53%3A57Z&sp=r'},\n",
              " 'submittedBy': 'Carlo Binda'}"
            ]
          },
          "execution_count": 9,
          "metadata": {},
          "output_type": "execute_result"
        }
      ],
      "source": [
        "# Create a script config\n",
        "script_config = ScriptRunConfig(source_directory=training_folder,\n",
        "                                script='diabetes_training.py',\n",
        "                                arguments = ['--reg_rate', 0.1],\n",
        "                                environment=env,\n",
        "                                docker_runtime_config=DockerConfiguration(use_docker=True)) \n",
        "\n",
        "# submit the experiment\n",
        "experiment_name = 'mslearn-train-diabetes'\n",
        "experiment = Experiment(workspace=ws, name=experiment_name)\n",
        "run = experiment.submit(config=script_config)\n",
        "RunDetails(run).show()\n",
        "run.wait_for_completion()"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Once again, we can get the metrics and outputs from the completed run."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 10,
      "metadata": {
        "gather": {
          "logged": 1649366082569
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "Regularization Rate 0.1\n",
            "Accuracy 0.774\n",
            "AUC 0.8483198169063138\n",
            "\n",
            "\n",
            "azureml-logs/60_control_log.txt\n",
            "azureml-logs/70_driver_log.txt\n",
            "logs/azureml/8_azureml.log\n",
            "outputs/diabetes_model.pkl\n"
          ]
        }
      ],
      "source": [
        "# Get logged metrics\n",
        "metrics = run.get_metrics()\n",
        "for key in metrics.keys():\n",
        "        print(key, metrics.get(key))\n",
        "print('\\n')\n",
        "for file in run.get_file_names():\n",
        "    print(file)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Register a new version of the model\n",
        "\n",
        "Now that you've trained a new model, you can register it as a new version in the workspace."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 11,
      "metadata": {
        "gather": {
          "logged": 1649366090545
        }
      },
      "outputs": [
        {
          "name": "stdout",
          "output_type": "stream",
          "text": [
            "diabetes_model version: 2\n",
            "\t Training context : Parameterized script\n",
            "\t AUC : 0.8483198169063138\n",
            "\t Accuracy : 0.774\n",
            "\n",
            "\n",
            "diabetes_model version: 1\n",
            "\t Training context : Script\n",
            "\t AUC : 0.8484929598487486\n",
            "\t Accuracy : 0.774\n",
            "\n",
            "\n"
          ]
        }
      ],
      "source": [
        "from azureml.core import Model\n",
        "\n",
        "# Register the model\n",
        "run.register_model(model_path='outputs/diabetes_model.pkl', model_name='diabetes_model',\n",
        "                   tags={'Training context':'Parameterized script'},\n",
        "                   properties={'AUC': run.get_metrics()['AUC'], 'Accuracy': run.get_metrics()['Accuracy']})\n",
        "\n",
        "# List registered models\n",
        "for model in Model.list(ws):\n",
        "    print(model.name, 'version:', model.version)\n",
        "    for tag_name in model.tags:\n",
        "        tag = model.tags[tag_name]\n",
        "        print ('\\t',tag_name, ':', tag)\n",
        "    for prop_name in model.properties:\n",
        "        prop = model.properties[prop_name]\n",
        "        print ('\\t',prop_name, ':', prop)\n",
        "    print('\\n')"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "You can also view registered models in your workspace on the **Models** page in [Azure Machine Learning studio](https://ml.azure.com).\n",
        "\n",
        "If you've finished exploring, you can close this notebook and shut down your compute instance."
      ]
    }
  ],
  "metadata": {
    "kernel_info": {
      "name": "python38-azureml"
    },
    "kernelspec": {
      "display_name": "Python 3.8 - AzureML",
      "language": "python",
      "name": "python38-azureml"
    },
    "language_info": {
      "codemirror_mode": {
        "name": "ipython",
        "version": 3
      },
      "file_extension": ".py",
      "mimetype": "text/x-python",
      "name": "python",
      "nbconvert_exporter": "python",
      "pygments_lexer": "ipython3",
      "version": "3.8.5 (default, Sep  4 2020, 07:30:14) \n[GCC 7.3.0]"
    },
    "nteract": {
      "version": "nteract-front-end@1.0.0"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 0
}