04B - Working with Datasets.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Working with Datasets\n",
    "\n",
    "In the previous labs, you used a *datastore* to provide centralized, cloud-based data access. In this lab, you'll explore *datasets*, a further abstraction that makes it easier to work with specific data for experiments and training.\n",
    "\n",
    "## Connect to Your Workspace\n",
    "\n",
    "The first thing you need to do is to connect to your workspace using the Azure ML SDK.\n",
    "\n",
    "> **Note**: If the authenticated session with your Azure subscription has expired since you completed the previous exercise, you'll be prompted to reauthenticate."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import azureml.core\n",
    "from azureml.core import Workspace\n",
    "\n",
    "# Load the workspace from the saved config file\n",
    "ws = Workspace.from_config()\n",
    "print('Ready to use Azure ML {} to work with {}'.format(azureml.core.VERSION, ws.name))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Prepare Data\n",
    "\n",
    "In the previous lab, you created a datastore. Datasets are usually (though not always) based on data in datastores.\n",
    "\n",
    "If you did not complete the previous lab, run the following code to upload two local CSV files to the default datastore in your workspace (if you *did* complete the previous lab, this will just overwrite the same files)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ws.get_default_datastore().upload_files(files=['./data/diabetes.csv', './data/diabetes2.csv'], # Upload the diabetes csv files in /data\n",
    "                       target_path='diabetes-data/', # Put it in a folder path in the datastore\n",
    "                       overwrite=True, # Replace existing files of the same name\n",
    "                       show_progress=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Create a Tabular Dataset\n",
    "\n",
    "A dataset is an object that encapsulates a specific data source. Let's create a dataset from the diabetes data you uploaded to the datastore, and view the first 20 records. In this case, the data is in a structured format in a CSV file, so we'll use a *Tabular* dataset."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Dataset\n",
    "\n",
    "# Get the default datastore\n",
    "default_ds = ws.get_default_datastore()\n",
    "\n",
    "#Create a tabular dataset from the path on the datastore (this may take a short while)\n",
    "tab_data_set = Dataset.Tabular.from_delimited_files(path=(default_ds, 'diabetes-data/*.csv'))\n",
    "\n",
    "# Display the first 20 rows as a Pandas dataframe\n",
    "tab_data_set.take(20).to_pandas_dataframe()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "As you can see in the code above, it's easy to convert a tabular dataset to a Pandas dataframe, enabling you to work with the data using common python techniques.\n",
    "\n",
    "## Create a File Dataset\n",
    "\n",
    "The dataset you created is a *tabular* dataset that can be read as a dataframe containing all of the data in the structured files that are included in the dataset definition. This works well for tabular data, but in some machine learning scenarios you might need to work with data that is unstructured; or you may simply want to handle reading the data from files in your own code. To accomplish this, you can use a *file* dataset, which creates a list of file paths in a virtual mount point, which you can use to read the data in the files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Create a file dataset from the path on the datastore (this may take a short while)\n",
    "file_data_set = Dataset.File.from_files(path=(default_ds, 'diabetes-data/*.csv'))\n",
    "\n",
    "# Get the files in the dataset\n",
    "for file_path in file_data_set.to_path():\n",
    "    print(file_path)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Register Datasets\n",
    "\n",
    "Now that you have created datasets that reference the diabetes data, you can register them to make them easily accessible to any experiment being run in the workspace.\n",
    "\n",
    "We'll register the tabular dataset as **diabetes dataset**, and the file dataset as **diabetes files**."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Register the tabular dataset\n",
    "tab_data_set = tab_data_set.register(workspace=ws, \n",
    "                           name='diabetes dataset',\n",
    "                           description='diabetes data',\n",
    "                           tags = {'format':'CSV'},\n",
    "                           create_new_version=True)\n",
    "\n",
    "# Register the file dataset\n",
    "file_data_set = file_data_set.register(workspace=ws, \n",
    "                           name='diabetes file dataset',\n",
    "                           description='diabetes files',\n",
    "                           tags = {'format':'CSV'},\n",
    "                           create_new_version=True)\n",
    "\n",
    "print('Datasets registered')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can view and manage datasets on the **Datasets** page for your workspace in the [Azure ML Studio web interface](https://ml.azure.com). You cal also get a list of datasets from the workspace object:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"Datasets:\")\n",
    "for dataset_name in list(ws.datasets.keys()):\n",
    "    dataset = Dataset.get_by_name(ws, dataset_name)\n",
    "    print(\"\\t\", dataset.name, 'version', dataset.version)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you completed Labs 2A and 2B, you will see that registered datasets include transformations created using the visual Designer tool. You may also notice that in registering **diabetes dataset** with the same name as the dataset you created using the *Studio* interface in a previous exercise, you are creating a new *version* of the dataset. The ability to version datasets enables you to redefine datasets without breaking existing experiments or pipelines that rely on previous definitions. By default, the latest version of a named dataset is returned, but you can retrieve a specific version of a dataset by specifying the version number, like this:\n",
    "\n",
    "```python\n",
    "dataset_v1 = Dataset.get_by_name(ws, 'diabetes dataset', version = 1)\n",
    "```"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "\n",
    "## Train a Model from a Tabular Dataset\n",
    "\n",
    "Now that you have datasets, you're ready to start training models from them. You can pass datasets to scripts as *inputs* in the estimator being used to run the script.\n",
    "\n",
    "Run the following two code cells to create:\n",
    "\n",
    "1. A folder named **diabetes_training_from_tab_dataset**\n",
    "2. A script that trains a classification model by using a tabular dataset that is passed to is as an *input*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Create a folder for the experiment files\n",
    "experiment_folder = 'diabetes_training_from_tab_dataset'\n",
    "os.makedirs(experiment_folder, exist_ok=True)\n",
    "print(experiment_folder, 'folder created')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile $experiment_folder/diabetes_training.py\n",
    "# Import libraries\n",
    "import argparse\n",
    "from azureml.core import Run\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import joblib\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "\n",
    "# Set regularization hyperparameter (passed as an argument to the script)\n",
    "parser = argparse.ArgumentParser()\n",
    "parser.add_argument('--regularization', type=float, dest='reg_rate', default=0.01, help='regularization rate')\n",
    "args = parser.parse_args()\n",
    "reg = args.reg_rate\n",
    "\n",
    "# Get the experiment run context\n",
    "run = Run.get_context()\n",
    "\n",
    "# load the diabetes data (passed as an input dataset)\n",
    "print(\"Loading Data...\")\n",
    "diabetes = run.input_datasets['diabetes'].to_pandas_dataframe()\n",
    "\n",
    "# Separate features and labels\n",
    "X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values\n",
    "\n",
    "# Split data into training set and test set\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)\n",
    "\n",
    "# Train a logistic regression model\n",
    "print('Training a logistic regression model with regularization rate of', reg)\n",
    "run.log('Regularization Rate',  np.float(reg))\n",
    "model = LogisticRegression(C=1/reg, solver=\"liblinear\").fit(X_train, y_train)\n",
    "\n",
    "# calculate accuracy\n",
    "y_hat = model.predict(X_test)\n",
    "acc = np.average(y_hat == y_test)\n",
    "print('Accuracy:', acc)\n",
    "run.log('Accuracy', np.float(acc))\n",
    "\n",
    "# calculate AUC\n",
    "y_scores = model.predict_proba(X_test)\n",
    "auc = roc_auc_score(y_test,y_scores[:,1])\n",
    "print('AUC: ' + str(auc))\n",
    "run.log('AUC', np.float(auc))\n",
    "\n",
    "os.makedirs('outputs', exist_ok=True)\n",
    "# note file saved in the outputs folder is automatically uploaded into experiment record\n",
    "joblib.dump(value=model, filename='outputs/diabetes_model.pkl')\n",
    "\n",
    "run.complete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now you can create an estimator to run the script, and define a named *input* for the training dataset, which is read by the script.\n",
    "\n",
    "> **Note**: The **Dataset** class is defined in the **azureml-dataprep** package (which is installed with the SDK), and this package includes optional support for **pandas** (which is used by the **to_pandas_dataframe()** method, so you need to include this package in the environment where the training experiment will be run."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.train.sklearn import SKLearn\n",
    "from azureml.core import Experiment\n",
    "from azureml.widgets import RunDetails\n",
    "\n",
    "# Set the script parameters\n",
    "script_params = {\n",
    "    '--regularization': 0.1\n",
    "}\n",
    "\n",
    "# Get the training dataset\n",
    "diabetes_ds = ws.datasets.get(\"diabetes dataset\")\n",
    "\n",
    "# Create an estimator\n",
    "estimator = SKLearn(source_directory=experiment_folder,\n",
    "                    entry_script='diabetes_training.py',\n",
    "                    script_params=script_params,\n",
    "                    compute_target = 'local',\n",
    "                    inputs=[diabetes_ds.as_named_input('diabetes')], # Pass the Dataset object as an input...\n",
    "                    pip_packages=['azureml-dataprep[pandas]'] # ...so you need the dataprep package\n",
    "                   )\n",
    "\n",
    "# Create an experiment\n",
    "experiment_name = 'diabetes-training'\n",
    "experiment = Experiment(workspace = ws, name = experiment_name)\n",
    "\n",
    "# Run the experiment\n",
    "run = experiment.submit(config=estimator)\n",
    "# Show the run details while running\n",
    "RunDetails(run).show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The first time the experiment is run, it may take some time to set up the Python environment - subsequent runs will be quicker.\n",
    "\n",
    "When the experiment has completed, in the widget, view the **azureml-logs/70_driver_log.txt** output log and the metrics generated by the run.\n",
    "\n",
    "As with all experiments, you can view the details of the experiment run in the [Azure ML Studio web interface](https://ml.azure.com), and you can write code to retrieve the metrics and files generated:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get logged metrics\n",
    "metrics = run.get_metrics()\n",
    "for key in metrics.keys():\n",
    "        print(key, metrics.get(key))\n",
    "print('\\n')\n",
    "for file in run.get_file_names():\n",
    "    print(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The model we trained is saved as the **diabetes_model.pkl** file in the **outputs** folder, so you can register it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Model\n",
    "\n",
    "run.register_model(model_path='outputs/diabetes_model.pkl', model_name='diabetes_model',\n",
    "                   tags={'Training context':'SKLearn Estimator (tabular dataset)'}, properties={'AUC': run.get_metrics()['AUC'], 'Accuracy': run.get_metrics()['Accuracy']})\n",
    "\n",
    "for model in Model.list(ws):\n",
    "    print(model.name, 'version:', model.version)\n",
    "    for tag_name in model.tags:\n",
    "        tag = model.tags[tag_name]\n",
    "        print ('\\t',tag_name, ':', tag)\n",
    "    for prop_name in model.properties:\n",
    "        prop = model.properties[prop_name]\n",
    "        print ('\\t',prop_name, ':', prop)\n",
    "    print('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Train a Model from a File Dataset\n",
    "\n",
    "You've seen how to train a model using training data in a *tabular* dataset; but what about a *file* dataset?\n",
    "\n",
    "When you;re using a file dataset, the dataset input passed to the script represents a mount point containing file paths. How you read the data from these files depends on the kind of data in the files and what you want to do with it. In the case of the diabetes CSV files, you can use the Python **glob** module to create a list of files in the virtual mount point defined by the dataset, and read them all into Pandas dataframes that are concatenated into a single dataframe.\n",
    "\n",
    "Run the following two code cells to create:\n",
    "\n",
    "1. A folder named **diabetes_training_from_file_dataset**\n",
    "2. A script that trains a classification model by using a file dataset that is passed to is as an *input*."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "# Create a folder for the experiment files\n",
    "experiment_folder = 'diabetes_training_from_file_dataset'\n",
    "os.makedirs(experiment_folder, exist_ok=True)\n",
    "print(experiment_folder, 'folder created')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%writefile $experiment_folder/diabetes_training.py\n",
    "# Import libraries\n",
    "import argparse\n",
    "from azureml.core import Workspace, Dataset, Experiment, Run\n",
    "import pandas as pd\n",
    "import numpy as np\n",
    "import joblib\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.linear_model import LogisticRegression\n",
    "from sklearn.metrics import roc_auc_score\n",
    "from sklearn.metrics import roc_curve\n",
    "import glob\n",
    "\n",
    "# Set regularization hyperparameter (passed as an argument to the script)\n",
    "parser = argparse.ArgumentParser()\n",
    "parser.add_argument('--regularization', type=float, dest='reg_rate', default=0.01, help='regularization rate')\n",
    "args = parser.parse_args()\n",
    "reg = args.reg_rate\n",
    "\n",
    "# Get the experiment run context\n",
    "run = Run.get_context()\n",
    "\n",
    "# load the diabetes dataset\n",
    "print(\"Loading Data...\")\n",
    "data_path = run.input_datasets['diabetes'] # Get the training data from the estimator input\n",
    "all_files = glob.glob(data_path + \"/*.csv\")\n",
    "diabetes = pd.concat((pd.read_csv(f) for f in all_files))\n",
    "\n",
    "# Separate features and labels\n",
    "X, y = diabetes[['Pregnancies','PlasmaGlucose','DiastolicBloodPressure','TricepsThickness','SerumInsulin','BMI','DiabetesPedigree','Age']].values, diabetes['Diabetic'].values\n",
    "\n",
    "# Split data into training set and test set\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, test_size=0.30, random_state=0)\n",
    "\n",
    "# Train a logistic regression model\n",
    "print('Training a logistic regression model with regularization rate of', reg)\n",
    "run.log('Regularization Rate',  np.float(reg))\n",
    "model = LogisticRegression(C=1/reg, solver=\"liblinear\").fit(X_train, y_train)\n",
    "\n",
    "# calculate accuracy\n",
    "y_hat = model.predict(X_test)\n",
    "acc = np.average(y_hat == y_test)\n",
    "print('Accuracy:', acc)\n",
    "run.log('Accuracy', np.float(acc))\n",
    "\n",
    "# calculate AUC\n",
    "y_scores = model.predict_proba(X_test)\n",
    "auc = roc_auc_score(y_test,y_scores[:,1])\n",
    "print('AUC: ' + str(auc))\n",
    "run.log('AUC', np.float(auc))\n",
    "\n",
    "os.makedirs('outputs', exist_ok=True)\n",
    "# note file saved in the outputs folder is automatically uploaded into experiment record\n",
    "joblib.dump(value=model, filename='outputs/diabetes_model.pkl')\n",
    "\n",
    "run.complete()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next we need to change the way we pass the dataset to the estimator - it needs to define a mount point from which the script can read the files. For large volumes of data, you'd generally use the **as_mount** method to stream the files directly from the dataset source; but when running on local compute (as we are in this example), you need to use the **as_download** option to download the dataset files to a local folder.\n",
    "\n",
    "Also, since the **Dataset** class is defined in the **azureml-dataprep** package, we need to include that in the experiment environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "from azureml.train.sklearn import SKLearn\n",
    "from azureml.core import Experiment\n",
    "from azureml.widgets import RunDetails\n",
    "\n",
    "# Set the script parameters\n",
    "script_params = {\n",
    "    '--regularization': 0.1\n",
    "}\n",
    "\n",
    "# Get the training dataset\n",
    "diabetes_ds = ws.datasets.get(\"diabetes file dataset\")\n",
    "\n",
    "# Create an estimator\n",
    "estimator = SKLearn(source_directory=experiment_folder,\n",
    "                    entry_script='diabetes_training.py',\n",
    "                    script_params=script_params,\n",
    "                    compute_target = 'local',\n",
    "                    inputs=[diabetes_ds.as_named_input('diabetes').as_download(path_on_compute='diabetes_data')], # Pass the Dataset object as an input\n",
    "                    pip_packages=['azureml-dataprep[pandas]'] # so we need the dataprep package\n",
    "                   )\n",
    "\n",
    "# Create an experiment\n",
    "experiment_name = 'diabetes-training'\n",
    "experiment = Experiment(workspace = ws, name = experiment_name)\n",
    "\n",
    "# Run the experiment\n",
    "run = experiment.submit(config=estimator)\n",
    "# Show the run details while running\n",
    "RunDetails(run).show()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "When the experiment has completed, in the widget, view the **azureml-logs/70_driver_log.txt** output log to verify that the file dataset was processed and the data files downloaded.\n",
    "\n",
    "As with all experiments, you can view the details of the experiment run in the [Azure ML Studio web interface](https://ml.azure.com), and you can write code to retrieve the metrics and files generated:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Get logged metrics\n",
    "metrics = run.get_metrics()\n",
    "for key in metrics.keys():\n",
    "        print(key, metrics.get(key))\n",
    "print('\\n')\n",
    "for file in run.get_file_names():\n",
    "    print(file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once again, let's register the model that we trained."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from azureml.core import Model\n",
    "\n",
    "run.register_model(model_path='outputs/diabetes_model.pkl', model_name='diabetes_model',\n",
    "                   tags={'Training context':'SKLearn Estimator (file dataset)'}, properties={'AUC': run.get_metrics()['AUC'], 'Accuracy': run.get_metrics()['Accuracy']})\n",
    "\n",
    "for model in Model.list(ws):\n",
    "    print(model.name, 'version:', model.version)\n",
    "    for tag_name in model.tags:\n",
    "        tag = model.tags[tag_name]\n",
    "        print ('\\t',tag_name, ':', tag)\n",
    "    for prop_name in model.properties:\n",
    "        prop = model.properties[prop_name]\n",
    "        print ('\\t',prop_name, ':', prop)\n",
    "    print('\\n')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "> **More Information**: For more information about training with datasets, see [Training with Datasets](https://docs.microsoft.com/en-us/azure/machine-learning/service/how-to-train-with-datasets) in the Azure ML documentation."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3.6 - AzureML",
   "language": "python",
   "name": "python3-azureml"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}