machine-learning/E2E-Text-Classification-API-with-Yelp-Dataset.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "# Text Classification API (preview)"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## What is text classification?\n",
        "\n",
        "Text classification as the name implies is the process of applying labels or categories to text.\n",
        "\n",
        "Common use cases include:\n",
        "\n",
        "- Categorizing e-mail as spam or not spam\n",
        "- Analyzing sentiment as positive or negative from customer reviews\n",
        "- Applying labels to support tickets\n",
        "\n",
        "## Solving text classification with machine learning\n",
        "\n",
        "Classification is a common problem in machine learning. There are a variety of algorithms you can use to train a classification model. Text classification is a subcategory of classification which deals specifically with raw text. Text poses interesting challenges because you have to account for the context and semantics in which the text occurs. As such, encoding meaning and context can be difficult. In recent years, deep learning models have emerged as a promising technique to solve natural language problems. More specifically, a type of neural network known as transformers has become the predominant way of solving natural language problems like text classification, translation, summarization, and question answering.\n",
        "\n",
        "Transformers were introduced in the paper [Attention is all you need](https://arxiv.org/abs/1706.03762).  Some popular transformer architectures for natural language tasks include:\n",
        "\n",
        "- Bidirectional Encoder Representations from Transformers (BERT)\n",
        "- Robustly Optimized BERT Pretraining Approach (RoBERTa)\n",
        "- Generative Pre-trained Transformer 2 (GPT-2)\n",
        "- Generative Pre-trained Transformer 3 (GPT-3)\n",
        "\n",
        "At a high level, transformers are a model architecture consisting of encoding and decoding layers. The encoder takes raw text as input and maps the input to a numerical representation (including context) to produce features. The decoder uses information from the encoder to produce output such as a category or label in the case of text classification. What makes these layers so special is the concept of attention. Attention is the idea of focusing on specific parts of an input based on the importance of their context in relation to other inputs in a sequence. For example, let's say I'm categorizing news articles based on the headline. Not all words in the headline are relevant. In a headline like \"Auto sales are at an all-time high\", a word like \"sales\" might get more attention and lead to labeling the article as business or finance.  \n",
        "\n",
        "Like most neural networks, training transformers from scratch can be expensive because they require large amounts of data and compute. However, you don't always have to train from scratch. Using a technique known as fine-tuning you can take a pre-trained model and retrain the layers specific to your domain or problem using your own data. This gives you the benefit of having a model that's more tailored to solve your problem without having to go through the process of training the entire model from scratch.  \n",
        "\n",
        "## The Text Classification API (preview)\n",
        "\n",
        "Now that you have a general overview of how text classification problems can be solved using deep learning, let's take a look at how we've incorporated many of these techniques into the Text Classification API.\n",
        "\n",
        "The Text Classification API is powered by [TorchSharp](https://github.com/dotnet/TorchSharp). TorchSharp is a .NET library that provides access to libtorch, the library that powers PyTorch. TorchSharp contains the building blocks for training neural networks from scratch in .NET. The TorchSharp components however are low-level and building neural networks from scratch has a steep learning curve. In ML.NET, we've abstracted some of that complexity to the scenario level."
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Install packages\n",
        "\n",
        "To use the Text Classification API, you'll have to install the following packages\n",
        "\n",
        "- [`Microsoft.ML`](https://www.nuget.org/packages/Microsoft.ML/)\n",
        "- [`Microsoft.ML.TorchSharp`](https://www.nuget.org/packages/Microsoft.ML.TorchSharp/)\n",
        "- [`TorchSharp-cpu`](https://www.nuget.org/packages/TorchSharp-cpu/) if you're using a CPU or [`TorchSharp-cuda-windows`](https://www.nuget.org/packages/TorchSharp-cuda-windows/) / [`TorchSharp-cuda-linux`](https://www.nuget.org/packages/TorchSharp-cuda-linux/) if you're using a GPU.\n",
        "\n",
        "To enable GPU support, you'll also have to install the CUDA dependencies. For more information, see the [GPU support guide](https://docs.microsoft.com/dotnet/machine-learning/how-to-guides/install-gpu-model-builder#install-dependencies)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        }
      },
      "source": [
        "#i \"nuget:https://pkgs.dev.azure.com/dnceng/public/_packaging/MachineLearning/nuget/v3/index.json\"\n",
        "\n",
        "#r \"nuget:Microsoft.ML,2.0.0-preview.22324.1\"\n",
        "#r \"nuget:Microsoft.ML.TorchSharp,0.20.0-preview.22324.1\"\n",
        "#r \"nuget:TorchSharp-cpu,0.96.7\"\n",
        "#r \"nuget:Microsoft.Data.Analysis,0.20.0-preview.22324.1\""
      ],
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": "<div><div><strong>Restore sources</strong><ul><li><span>https://pkgs.dev.azure.com/dnceng/public/_packaging/MachineLearning/nuget/v3/index.json</span></li></ul></div><div></div><div><strong>Installed Packages</strong><ul><li><span>Microsoft.Data.Analysis, 0.20.0-preview.22324.1</span></li><li><span>Microsoft.ML, 2.0.0-preview.22324.1</span></li><li><span>Microsoft.ML.TorchSharp, 0.20.0-preview.22324.1</span></li><li><span>TorchSharp-cpu, 0.96.7</span></li></ul></div></div>"
          },
          "execution_count": 1,
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/markdown": "Loading extensions from `Microsoft.Data.Analysis.Interactive.dll`"
          },
          "execution_count": 1,
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Add using statements"
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        }
      },
      "source": [
        "using Microsoft.ML;\n",
        "using Microsoft.ML.Data;\n",
        "using Microsoft.Data.Analysis;\n",
        "using Microsoft.ML.TorchSharp;"
      ],
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Initialize MLContext\n",
        "\n",
        "All ML.NET operations start in the MLContext class. Initializing mlContext creates a new ML.NET environment that can be shared across the model creation workflow objects. It's similar, conceptually, to DBContext in Entity Framework."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        }
      },
      "source": [
        "var mlContext = new MLContext();"
      ],
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "### Download or Locate Data\n",
        "The following code tries to locate the data file in a few known locations or it will download it from the known GitHub location."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        }
      },
      "source": [
        "using System;\n",
        "using System.IO;\n",
        "using System.Net;\n",
        "\n",
        "string EnsureDataSetDownloaded(string fileName)\n",
        "{\n",
        "\n",
        "\t// This is the path if the repo has been checked out.\n",
        "\tvar filePath = Path.Combine(Directory.GetCurrentDirectory(),\"data\", fileName);\n",
        "\n",
        "\tif (!File.Exists(filePath))\n",
        "\t{\n",
        "\t\t// This is the path if the file has already been downloaded.\n",
        "\t\tfilePath = Path.Combine(Directory.GetCurrentDirectory(), fileName);\n",
        "\t}\n",
        "\n",
        "\tif (!File.Exists(filePath))\n",
        "\t{\n",
        "\t\tusing (var client = new WebClient())\n",
        "\t\t{\n",
        "\t\t\tclient.DownloadFile($\"https://raw.githubusercontent.com/dotnet/csharp-notebooks/main/machine-learning/data/{fileName}\", filePath);\n",
        "\t\t}\n",
        "\t\tConsole.WriteLine($\"Downloaded {fileName}  to : {filePath}\");\n",
        "\t}\n",
        "\telse\n",
        "\t{\n",
        "\t\tConsole.WriteLine($\"{fileName} found here: {filePath}\");\n",
        "\t}\n",
        "\n",
        "\treturn filePath;\n",
        "}"
      ],
      "outputs": []
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        }
      },
      "source": [
        "var yelp_reviews = EnsureDataSetDownloaded(\"yelp_labelled.txt\");\n",
        "var columnNames = new [] {\"Text\", \"Sentiment\"};\n",
        "var df = DataFrame.LoadCsvFromString(yelp_reviews, separator:'\\t',header:false, columnNames:columnNames);"
      ],
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": "yelp_labelled.txt found here: C:\\dev\\csharp-notebooks\\machine-learning\\data\\yelp_labelled.txt\r\n"
          },
          "execution_count": 1,
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Once the data is loaded, use the `Head` method to preview the first three rows."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        }
      },
      "source": [
        "df.Head(3)"
      ],
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": "<table id=\"table_637919371773035870\"><thead><tr><th><i>index</i></th><th>Text</th><th>Sentiment</th></tr></thead><tbody><tr><td><i><div class=\"dni-plaintext\">0</div></i></td><td>Wow... Loved this place.</td><td><div class=\"dni-plaintext\">1</div></td></tr><tr><td><i><div class=\"dni-plaintext\">1</div></i></td><td>Crust is not good.</td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td><i><div class=\"dni-plaintext\">2</div></i></td><td>Not tasty and the texture was just nasty.</td><td><div class=\"dni-plaintext\">0</div></td></tr></tbody></table>"
          },
          "execution_count": 1,
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "> The datasets this tutorial uses a dataset from the 'From Group to Individual Labels using Deep Features', Kotzias et al,. KDD 2015, and hosted at the UCI Machine Learning Repository - Dua, D. and Karra Taniskidou, E. (2017). [UCI Machine Learning Repository](http://archive.ics.uci.edu/ml). Irvine, CA: University of California, School of Information and Computer Science.\n",
        "\n",
        "The dataset contains two columns:\n",
        "\n",
        "- **Text:** The raw review text from Yelp\n",
        "- **Sentiment:** A binary value to represent the sentiment of the review. 0 is negative and 1 is positive. "
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Split the data into train and test sets. \n",
        "\n",
        "The original dataset is split into two subsets: train and test. The train set is what you'll use to learn the patterns of your data. The test set is used to evaluate the performance of your model using evaluation metrics for the classification task.\n",
        "\n",
        "In this case, 80% of the data is used for training as defined by the `testFraction` parameter. The remaining 20% is used for evaluation and testing."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        }
      },
      "source": [
        "var trainTestSplit = mlContext.Data.TrainTestSplit(df, testFraction:0.2);"
      ],
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Define your training pipeline\n",
        "\n",
        "The Text Classification API is part of the multiclass classification catalog. To use it, add the `TextClassification` trainer to your pipeline. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        }
      },
      "source": [
        "var pipeline =\n",
        "\t\tmlContext.Transforms.Conversion.MapValueToKey(\"Label\",\"Sentiment\")\n",
        "\t\t\t.Append(mlContext.MulticlassClassification.Trainers.TextClassification(sentence1ColumnName: \"Text\"))\n",
        "\t\t\t.Append(mlContext.Transforms.Conversion.MapKeyToValue(\"PredictedLabel\"));"
      ],
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Train the model\n",
        "\n",
        "Use the training dataset to train your model using the `Fit` method."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        }
      },
      "source": [
        "var model = pipeline.Fit(trainTestSplit.TrainSet);"
      ],
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Use the model to make predictions\n",
        "\n",
        "Use your model to make predictions on the test dataset by calling the `Transform` method. "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        }
      },
      "source": [
        "var predictionIDV = model.Transform(trainTestSplit.TestSet);"
      ],
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "The result of calling `Transform` is an `IDataView` with your predicted values. To make it easier to view your predictions, convert the `IDataView` to an `IDataFrame` . In this case, the only columns that I'm interested in are the Text, Sentiment (actual value), and PredictedLabel (predicted value). "
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        }
      },
      "source": [
        "var columnsToSelect = new [] {\"Text\", \"Sentiment\", \"PredictedLabel\"};\n",
        "\n",
        "var predictions = predictionIDV.ToDataFrame(columnsToSelect);"
      ],
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Use the `Tail` method to preview the last three rows in your prediction `DataFrame`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        }
      },
      "source": [
        "predictions.Tail(3)"
      ],
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": "<table id=\"table_637919373979111360\"><thead><tr><th><i>index</i></th><th>Text</th><th>Sentiment</th><th>PredictedLabel</th></tr></thead><tbody><tr><td><i><div class=\"dni-plaintext\">0</div></i></td><td>Oh this is such a thing of beauty, this restaurant.</td><td><div class=\"dni-plaintext\">1</div></td><td><div class=\"dni-plaintext\">0</div></td></tr><tr><td><i><div class=\"dni-plaintext\">1</div></i></td><td>A greasy, unhealthy meal.</td><td><div class=\"dni-plaintext\">0</div></td><td><div class=\"dni-plaintext\">1</div></td></tr><tr><td><i><div class=\"dni-plaintext\">2</div></i></td><td>The best place in Vegas for breakfast (just check out a Sat, or Sun.</td><td><div class=\"dni-plaintext\">1</div></td><td><div class=\"dni-plaintext\">1</div></td></tr></tbody></table>"
          },
          "execution_count": 1,
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "## Evaluate the model"
      ]
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "There's a variety of metrics you can use to evaluate how well your model performs.  Use the [Evaluate](https://docs.microsoft.com/dotnet/api/microsoft.ml.multiclassclassificationcatalog.evaluate?view=ml-dotnet) method to calculate the evaluation metrics for your model using the predictions `IDataView`."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        }
      },
      "source": [
        "var evaluationMetrics = \n",
        "\tmlContext\n",
        "\t\t.MulticlassClassification\n",
        "\t\t.Evaluate(predictionIDV);"
      ],
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "metadata": {},
      "source": [
        "Then, display the evaluation metrics. For more information on multiclass classification evaluation metrics, see the [ML.NET evaluation metrics guide](https://docs.microsoft.com/dotnet/machine-learning/resources/metrics#evaluation-metrics-for-multi-class-classification)."
      ]
    },
    {
      "cell_type": "code",
      "execution_count": 1,
      "metadata": {
        "dotnet_interactive": {
          "language": "csharp"
        }
      },
      "source": [
        "evaluationMetrics"
      ],
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/html": "<table><thead><tr><th>LogLoss</th><th>LogLossReduction</th><th>MacroAccuracy</th><th>MicroAccuracy</th><th>TopKAccuracy</th><th>TopKPredictionCount</th><th>TopKAccuracyForAllK</th><th>PerClassLogLoss</th><th>ConfusionMatrix</th></tr></thead><tbody><tr><td><div class=\"dni-plaintext\">10.53512863047496</div></td><td><div class=\"dni-plaintext\">-14.199291365827746</div></td><td><div class=\"dni-plaintext\">0.6737016700983757</div></td><td><div class=\"dni-plaintext\">0.6737967914438503</div></td><td><div class=\"dni-plaintext\">0</div></td><td><div class=\"dni-plaintext\">0</div></td><td><div class=\"dni-plaintext\">&lt;null&gt;</div></td><td><div class=\"dni-plaintext\">[ 10.940300196581468, 10.134267400178105 ]</div></td><td><div class=\"dni-plaintext\">{ Microsoft.ML.Data.ConfusionMatrix: PerClassPrecision: [ 0.6777777777777778, 0.6701030927835051 ], PerClassRecall: [ 0.6559139784946236, 0.6914893617021277 ], Counts: [ [ 61, 32 ], [ 29, 65 ] ], NumberOfClasses: 2 }</div></td></tr></tbody></table>"
          },
          "execution_count": 1,
          "metadata": {}
        }
      ]
    }
  ],
  "metadata": {
    "kernelspec": {
      "display_name": ".NET (C#)",
      "language": "C#",
      "name": ".net-csharp"
    },
    "language_info": {
      "file_extension": ".cs",
      "mimetype": "text/x-csharp",
      "name": "C#",
      "pygments_lexer": "csharp",
      "version": "8.0"
    }
  },
  "nbformat": 4,
  "nbformat_minor": 4
}