machine-learning/Parameter and SearchSpace.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "dotnet_interactive": {
     "language": "markdown"
    }
   },
   "source": [
    "In AutoML.Net, [`SearchSpace`](https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.searchspace.searchspace?view=ml-dotnet-preview) defines the hyper-parameter searching range of a sweepable pipeline or a sweepable estimator and [`Parameter`](https://docs.microsoft.com/en-us/dotnet/api/microsoft.ml.searchspace.parameter?view=ml-dotnet-preview) is the sampled result from a `SearchSpace`, which can be used to restore an estimator or a pipeline.\n",
    "\n",
    "`SearchSpace` has a great impact on automl performance. In theory, it decides the upper-bound of an automl model. A larger `SearchSpace` usually means an increasing potential of finding a better model. In practice though, there's also a trade-off and a larger `SearchSpace` doesn't always to be better considering the increasing searching complexity and higher training cost.\n",
    "\n",
    "AutoML.Net provides default `SearchSpace` for almost all available ML.Net trainers, which is well-tuned on hundreds of dataset and should be good enough in 99% situation.  However, you can also provide your own search space if the default one doesn't meet your request.\n",
    "\n",
    "In this notebook, we will go through the following topics on search space\n",
    "- default search space for mlnet trainers.\n",
    "- how to customize search space\n",
    "- how to sample from search space"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Install nuget dependency and import using statement"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "dotnet-interactive.csharp"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div><div><strong>Restore sources</strong><ul><li><span>https://pkgs.dev.azure.com/dnceng/public/_packaging/MachineLearning/nuget/v3/index.json</span></li></ul></div><div></div><div><strong>Installed Packages</strong><ul><li><span>Microsoft.Data.Analysis, 0.20.0-preview.22356.1</span></li><li><span>Microsoft.ML.AutoML, 0.20.0-preview.22356.1</span></li><li><span>Plotly.NET.CSharp, 0.0.1</span></li><li><span>Plotly.NET.Interactive, 3.0.2</span></li></ul></div></div>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Loading extensions from `Microsoft.ML.AutoML.Interactive.dll`"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Loading extensions from `Microsoft.Data.Analysis.Interactive.dll`"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "data": {
      "text/markdown": [
       "Loading extensions from `Plotly.NET.Interactive.dll`"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "// install dependencies and import using statement\n",
    "#i \"nuget:https://pkgs.dev.azure.com/dnceng/public/_packaging/MachineLearning/nuget/v3/index.json\"\n",
    "#r \"nuget: Plotly.NET.Interactive, 3.0.2\"\n",
    "#r \"nuget: Plotly.NET.CSharp, 0.0.1\"\n",
    "\n",
    "// make sure you are using Microsoft.ML.AutoML later than 0.20.0.\n",
    "#r \"nuget: Microsoft.ML.AutoML, 0.20.0-preview.22356.1\"\n",
    "#r \"nuget: Microsoft.Data.Analysis, 0.20.0-preview.22356.1\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "dotnet-interactive.csharp"
    }
   },
   "outputs": [],
   "source": [
    "// Import usings.\n",
    "using Microsoft.Data.Analysis;\n",
    "using System;\n",
    "using System.IO;\n",
    "using Microsoft.ML;\n",
    "using Microsoft.ML.AutoML;\n",
    "using Microsoft.ML.Data;\n",
    "using Microsoft.ML.SearchSpace;\n",
    "using Newtonsoft.Json;\n",
    "using Microsoft.ML.AutoML.CodeGen;\n",
    "using Microsoft.ML.SearchSpace.Option;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Default search space for ml.net trainer\n",
    "AutoML.Net comes with a series of default search space for most of ml.net trainers. You can check it under `Microsoft.ML.AutoML.CodeGen` namespace. The following code shows the default search space for LightGbm.\n",
    "\n",
    "The default search space will be used in `AutoFeaturizer` and `AutoTrainer` api, which provides an easy way to create sweepable estimators."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "dotnet-interactive.csharp"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{\r\n",
       "  \"NumberOfLeaves\": {\r\n",
       "    \"Min\": 4.0,\r\n",
       "    \"Max\": 1024.0,\r\n",
       "    \"LogBase\": true,\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      null\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      0.0\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"MinimumExampleCountPerLeaf\": {\r\n",
       "    \"Min\": 20.0,\r\n",
       "    \"Max\": 1024.0,\r\n",
       "    \"LogBase\": true,\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      null\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      0.0\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"LearningRate\": {\r\n",
       "    \"Min\": 2E-10,\r\n",
       "    \"Max\": 1.0,\r\n",
       "    \"LogBase\": true,\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      null\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      1.0\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"NumberOfTrees\": {\r\n",
       "    \"Min\": 4.0,\r\n",
       "    \"Max\": 32768.0,\r\n",
       "    \"LogBase\": true,\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      null\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      0.0\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"SubsampleFraction\": {\r\n",
       "    \"Min\": 2E-10,\r\n",
       "    \"Max\": 1.0,\r\n",
       "    \"LogBase\": true,\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      null\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      1.0\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"MaximumBinCountPerFeature\": {\r\n",
       "    \"Min\": 8.0,\r\n",
       "    \"Max\": 1024.0,\r\n",
       "    \"LogBase\": true,\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      null\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      0.7142857142857141\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"FeatureFraction\": {\r\n",
       "    \"Min\": 2E-10,\r\n",
       "    \"Max\": 1.0,\r\n",
       "    \"LogBase\": false,\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      null\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      1.0\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"L1Regularization\": {\r\n",
       "    \"Min\": 2E-10,\r\n",
       "    \"Max\": 1.0,\r\n",
       "    \"LogBase\": true,\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      null\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      0.0\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"L2Regularization\": {\r\n",
       "    \"Min\": 2E-10,\r\n",
       "    \"Max\": 1.0,\r\n",
       "    \"LogBase\": true,\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      null\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      1.0\r\n",
       "    ]\r\n",
       "  }\r\n",
       "}"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "var lgbmSearchSpace = new SearchSpace<LgbmOption>();\n",
    "\n",
    "// the default option for NumberOfLeaves is (4, 32768, logBase: true)\n",
    "// this might be too large but you can refine it if necessary\n",
    "lgbmSearchSpace[\"NumberOfLeaves\"] = new UniformIntOption(4, 1024, true, 4);\n",
    "\n",
    "// pass lgbmSearchSpace in AutoTrainer\n",
    "var context = new MLContext();\n",
    "var pipeline = context.Auto().BinaryClassification(lgbmSearchSpace: lgbmSearchSpace);\n",
    "\n",
    "JsonConvert.SerializeObject(lgbmSearchSpace, Formatting.Indented)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    }
   },
   "source": [
    "# Create `SearchSpace` from scratch.\n",
    "The following code shows how to create a `SearchSpace` that contains numeric, choice and nested options, where\n",
    "- numeric option is an option that is numeric type, like float, double, int...\n",
    "- choice option is an option that is descrete, like string or boolean.\n",
    "- nested option is an option that itself is also a search space.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "dotnet-interactive.csharp"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{\r\n",
       "  \"IntOption\": {\r\n",
       "    \"Min\": -10.0,\r\n",
       "    \"Max\": 10.0,\r\n",
       "    \"LogBase\": false,\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      null\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      0.5\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"SingleOption\": {\r\n",
       "    \"Min\": 1.0,\r\n",
       "    \"Max\": 10.0,\r\n",
       "    \"LogBase\": true,\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      null\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      0.0\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"DoubleOption\": {\r\n",
       "    \"Min\": -10.0,\r\n",
       "    \"Max\": 10.0,\r\n",
       "    \"LogBase\": false,\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      null\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      0.5\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"BoolOption\": {\r\n",
       "    \"Choices\": [\r\n",
       "      false,\r\n",
       "      true\r\n",
       "    ],\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      2\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      0.0\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"StrOption\": {\r\n",
       "    \"Choices\": [\r\n",
       "      \"a\",\r\n",
       "      \"b\",\r\n",
       "      \"c\"\r\n",
       "    ],\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      3\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      0.0\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"Nest\": {\r\n",
       "    \"IntOption\": {\r\n",
       "      \"Min\": -10.0,\r\n",
       "      \"Max\": 10.0,\r\n",
       "      \"LogBase\": false,\r\n",
       "      \"FeatureSpaceDim\": 1,\r\n",
       "      \"Step\": [\r\n",
       "        null\r\n",
       "      ],\r\n",
       "      \"Default\": [\r\n",
       "        0.5\r\n",
       "      ]\r\n",
       "    }\r\n",
       "  }\r\n",
       "}"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "using Microsoft.ML.SearchSpace.Option;\n",
    "\n",
    "var searchSpace = new SearchSpace();\n",
    "\n",
    "// numeric options\n",
    "searchSpace[\"IntOption\"] = new UniformIntOption(-10, 10, false, 0);\n",
    "searchSpace[\"SingleOption\"] = new UniformSingleOption(1, 10, true, 1);\n",
    "searchSpace[\"DoubleOption\"] = new UniformDoubleOption(-10, 10, false, 0);\n",
    "\n",
    "// choice options\n",
    "searchSpace[\"BoolOption\"] = new ChoiceOption(true, false);\n",
    "searchSpace[\"StrOption\"] = new ChoiceOption(\"a\", \"b\", \"c\");\n",
    "\n",
    "// nest options\n",
    "var nestedSearchSpace = new SearchSpace();\n",
    "nestedSearchSpace[\"IntOption\"] = new UniformIntOption(-10, 10, false, 0);\n",
    "searchSpace[\"Nest\"] = nestedSearchSpace;\n",
    "\n",
    "// pretty print search space\n",
    "JsonConvert.SerializeObject(searchSpace, Formatting.Indented)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# create `SearchSpace<T>` which casts sampled parameter to a concrete type.\n",
    "AutoML allows you to define `SearchSpace` with class type, in which case it checks class property and build up search space using those properties with option attributions. The following code shows how to create an identical search space from above using except with a class type.\n",
    "\n",
    "`SearchSpace<T>`, comparing with `SearchSpace`, will also cast sampled parameter to a concrete type `T` instead of `Parameter`, which saves the effort of getting specific hyper-parameter value from `Parameter`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "dotnet-interactive.csharp"
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{\r\n",
       "  \"IntOption\": {\r\n",
       "    \"Min\": -10.0,\r\n",
       "    \"Max\": 10.0,\r\n",
       "    \"LogBase\": false,\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      null\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      0.5\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"SingleOption\": {\r\n",
       "    \"Min\": 1.0,\r\n",
       "    \"Max\": 10.0,\r\n",
       "    \"LogBase\": true,\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      null\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      0.0\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"DoubleOption\": {\r\n",
       "    \"Min\": -10.0,\r\n",
       "    \"Max\": 10.0,\r\n",
       "    \"LogBase\": false,\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      null\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      0.0\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"BoolOption\": {\r\n",
       "    \"Choices\": [\r\n",
       "      false,\r\n",
       "      true\r\n",
       "    ],\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      2\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      0.0\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"StrOption\": {\r\n",
       "    \"Choices\": [\r\n",
       "      \"a\",\r\n",
       "      \"b\",\r\n",
       "      \"c\"\r\n",
       "    ],\r\n",
       "    \"FeatureSpaceDim\": 1,\r\n",
       "    \"Step\": [\r\n",
       "      3\r\n",
       "    ],\r\n",
       "    \"Default\": [\r\n",
       "      0.0\r\n",
       "    ]\r\n",
       "  },\r\n",
       "  \"Nest\": {\r\n",
       "    \"IntOption\": {\r\n",
       "      \"Min\": -10.0,\r\n",
       "      \"Max\": 10.0,\r\n",
       "      \"LogBase\": false,\r\n",
       "      \"FeatureSpaceDim\": 1,\r\n",
       "      \"Step\": [\r\n",
       "        null\r\n",
       "      ],\r\n",
       "      \"Default\": [\r\n",
       "        0.5\r\n",
       "      ]\r\n",
       "    }\r\n",
       "  }\r\n",
       "}"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "public class MyParameter\n",
    "{\n",
    "    [Range((int)-10, 10, 0, false)]\n",
    "    public int IntOption {get; set;}\n",
    "\n",
    "    [Range(1f, 10f, 1f, true)]\n",
    "    public float SingleOption {get; set;}\n",
    "\n",
    "    [Range(-10, 10, false)]\n",
    "    public double DoubleOption {get; set;}\n",
    "\n",
    "    [BooleanChoice]\n",
    "    public bool BoolOption {get; set;}\n",
    "\n",
    "    [Choice(\"a\", \"b\", \"c\")]\n",
    "    public string StrOption {get; set;}\n",
    "\n",
    "    [NestOption]\n",
    "    public NestParameter Nest {get; set;}\n",
    "}\n",
    "\n",
    "public class NestParameter\n",
    "{\n",
    "    [Range((int)-10, 10, 0, false)]\n",
    "    public int IntOption {get; set;}\n",
    "}\n",
    "\n",
    "var searchSpace = new SearchSpace<MyParameter>();\n",
    "\n",
    "// pretty print search space\n",
    "JsonConvert.SerializeObject(searchSpace, Formatting.Indented)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Sampling `Parameter` from `SearchSpace`\n",
    "In AutoML.Net, `SearchSpace` is associated with an `n`-dimension vector which is called `feature space`. And tuner, instead of performing sampling on original options in a `SearchSpace`, sampling from that `feature space` instead. The sampling result will then mapping back to options.\n",
    "\n",
    "The following code shows the sampling process above. It creates an `n`-d random vector and uses that vector to sample from search space."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "dotnet_interactive": {
     "language": "csharp"
    },
    "vscode": {
     "languageId": "dotnet-interactive.csharp"
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0 - {\"IntOption\":5,\"SingleOption\":1.3184657,\"DoubleOption\":2.0,\"BoolOption\":true,\"StrOption\":\"b\",\"Nest\":{\"IntOption\":4}}\r\n",
      "0 - {\"IntOption\":5,\"SingleOption\":1.3184657,\"DoubleOption\":2.0,\"BoolOption\":false,\"StrOption\":\"b\",\"Nest\":{\"IntOption\":4}}\r\n",
      "1 - {\"IntOption\":4,\"SingleOption\":1.8871154,\"DoubleOption\":-7.0,\"BoolOption\":true,\"StrOption\":\"b\",\"Nest\":{\"IntOption\":1}}\r\n",
      "1 - {\"IntOption\":4,\"SingleOption\":1.8871154,\"DoubleOption\":-7.0,\"BoolOption\":false,\"StrOption\":\"b\",\"Nest\":{\"IntOption\":1}}\r\n",
      "2 - {\"IntOption\":-2,\"SingleOption\":5.2511787,\"DoubleOption\":5.0,\"BoolOption\":false,\"StrOption\":\"b\",\"Nest\":{\"IntOption\":6}}\r\n",
      "2 - {\"IntOption\":-2,\"SingleOption\":5.2511787,\"DoubleOption\":5.0,\"BoolOption\":false,\"StrOption\":\"b\",\"Nest\":{\"IntOption\":6}}\r\n"
     ]
    }
   ],
   "source": [
    "var rnd = new Random();\n",
    "SearchSpace searchSpace = new SearchSpace<MyParameter>();\n",
    "\n",
    "foreach(var i in Enumerable.Range(0, 3))\n",
    "{\n",
    "    var dim = searchSpace.FeatureSpaceDim;\n",
    "    var featureVector = Enumerable.Range(0, dim).Select(x => rnd.NextDouble()).ToArray();\n",
    "    var parameter = searchSpace.SampleFromFeatureSpace(featureVector);\n",
    "\n",
    "    // use Parameter.AsType<T> to map parameter to a concrete type.\n",
    "    var myParameter = parameter.AsType<MyParameter>();\n",
    "    var json = JsonConvert.SerializeObject(myParameter);\n",
    "\n",
    "    Console.WriteLine($\"{i} - {json}\");\n",
    "\n",
    "    // equivalant to\n",
    "    var intOption = parameter[\"IntOption\"].AsType<int>();\n",
    "    var singleOption = parameter[\"SingleOption\"].AsType<float>();\n",
    "    var doubleOption = parameter[\"DoubleOption\"].AsType<double>();\n",
    "    var strOption = parameter[\"StrOption\"].AsType<string>();\n",
    "    var nest = parameter[\"Nest\"].AsType<NestParameter>();\n",
    "\n",
    "    myParameter = new MyParameter()\n",
    "    {\n",
    "        IntOption = intOption,\n",
    "        SingleOption = singleOption,\n",
    "        DoubleOption = doubleOption,\n",
    "        StrOption = strOption,\n",
    "        Nest = nest,\n",
    "    };\n",
    "    json = JsonConvert.SerializeObject(myParameter);\n",
    "\n",
    "    Console.WriteLine($\"{i} - {json}\");\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Continue learning\n",
    "\n",
    "> [E2E-Forecasting using SSA with Luna Dataset](./E2E-Forecasting%20using%20SSA%20with%20Luna%20Dataset.ipynb) - This shows how to create `SweepableEstimator` with custom search space."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": ".NET (C#)",
   "language": "C#",
   "name": ".net-csharp"
  },
  "language_info": {
   "file_extension": ".cs",
   "mimetype": "text/x-csharp",
   "name": "C#",
   "pygments_lexer": "csharp",
   "version": "9.0"
  },
  "orig_nbformat": 4
 },
 "nbformat": 4,
 "nbformat_minor": 2
}