hf_trainer_bert_sentiment_target.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/hf_trainer_bert_sentiment_target.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Setup\n",
        "\n",
        "First, set up the required Python modules and perform some general configuration.\n",
        "\n",
        "(This part of the code follows the [MLP notebook](https://github.com/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/hf_trainer_mlp.ipynb) that you should already be familiar with.)"
      ],
      "metadata": {
        "id": "wo13ZXoZYB6J"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Install the required Python packages using [pip](https://en.wikipedia.org/wiki/Pip):\n",
        "\n",
        "* [`transformers`](https://huggingface.co/docs/transformers/index) is a popular deep learning package\n",
        "* [`datasets`](https://huggingface.co/docs/datasets/) provides support for loading, creating, and manipulating datasets\n",
        "\n",
        "Both of these packages are used extensively on this course."
      ],
      "metadata": {
        "id": "qKDDc9isYnxz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip3 install -q transformers datasets"
      ],
      "metadata": {
        "id": "4pquj9Xoxaza"
      },
      "execution_count": 68,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "(Above, the `!` at the start of the line tells the notebook to run the line as an operating system command rather than Python code, and the `-q` argument to `pip` runs the command in \"quiet\" mode, with less output.)"
      ],
      "metadata": {
        "id": "JyyZCbB0Yxmq"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We'll also use the [`pprint`](https://docs.python.org/3/library/pprint.html) (\"pretty-print\") module to format output more readably below. The only difference to just using `print` is that some data structures will be easier to read and interpret."
      ],
      "metadata": {
        "id": "N2oLE1FpY09U"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from pprint import PrettyPrinter\n",
        "\n",
        "pprint = PrettyPrinter(compact=True).pprint"
      ],
      "metadata": {
        "id": "cQ63zw6BY7tn"
      },
      "execution_count": 69,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Finally, we will reduce logging output. The `transformers` library by default produces fairly verbose logging. Commenting out the following code will enable low-priority output (`INFO` logging level and below)."
      ],
      "metadata": {
        "id": "xkrZ6e_FY_2e"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import logging\n",
        "\n",
        "logging.disable(logging.INFO)"
      ],
      "metadata": {
        "id": "ZH2bY_GjZG24"
      },
      "execution_count": 70,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---\n",
        "\n",
        "# Download and prepare data\n",
        "\n",
        "We will again use the `datasets` library function [`load_dataset`](https://huggingface.co/docs/datasets/master/en/package_reference/loading_methods#datasets.load_dataset) to load a dataset for our experiments.\n",
        "\n",
        "The data is in the TSV format you may recall from the [MLP notebook](https://github.com/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/hf_trainer_mlp.ipynb). We will prepare it here using the same process as in that notebook.\n",
        "\n",
        "First, download the TSV-formatted dataset:"
      ],
      "metadata": {
        "id": "W4x7GbT2ZKUJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!wget -nc https://raw.githubusercontent.com/TurkuNLP/sentiment-target-corpus/main/sentiment-target-fi.tsv"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "--hnFb7qzypC",
        "outputId": "0271a00d-f33b-4516-a9a9-eb7dd0399264"
      },
      "execution_count": 71,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "File ‘sentiment-target-fi.tsv’ already there; not retrieving.\n",
            "\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "(Above, the `-nc` argument tells wget not to download the file again if it is already found on the computer. This option is useful in particular if running a notebook more than once.)\n"
      ],
      "metadata": {
        "id": "42B6L_8Wz7wx"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's have a look at the first few lines of the file to get an idea of its contents.\n"
      ],
      "metadata": {
        "id": "1mHCRBfq0O8F"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "lines = []\n",
        "with open(\"sentiment-target-fi.tsv\") as f:    # open file\n",
        "    for line in f:    # iterate over lines in file\n",
        "        line = line.rstrip(\"\\n\")    # remove newline character\n",
        "        lines.append(line)    # store line in list\n",
        "\n",
        "for line in lines[:10]:    # iterate over first ten lines\n",
        "    print(line)    # print each line"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "qPNXh1DR0Q1d",
        "outputId": "f1ed711e-7175-433c-8e04-a7a4ec7c916d"
      },
      "execution_count": 72,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "#ID\ttext\tmajority\tagreement\tannotations\n",
            "comments2015d-45552-1.1\t<TARGET>Kuopionlahden</TARGET> vierasvenelaituri (kuten nimi oikeasti on) on mielestämme tosi kiva.\tpositive\t0.6666666666666666\tpositive,positive,neither,positive,neither,positive\n",
            "comments2014a-469816-3.1\tKäytetyssä <TARGET>Meganessa</TARGET> yksi etu verrattuna vaikkapa Skoda Octaviaan on se että Meganen voi ostaa jonkun vuoden uudempana.\tpositive\t1.0\tpositive,positive,positive,positive,positive,positive\n",
            "comments2007d-182055-6.1\tMonet heistä ovat olleet kymmeniä vuosia <TARGET>Finnairin</TARGET> palveluksessa ja nyt tulee kenkää.\tnegative\t0.8\tnegative,negative,negative,neither,negative\n",
            "threads2013a-835251-2.1\t<TARGET>Keskusta</TARGET> on aina ollut (ja on oleva) muun Suomen tuki ja turva.\tpositive\t1.0\tpositive,positive,positive\n",
            "comments2009a-483243-7.1\tItselläni on vuosimallin -93 <TARGET>LS-400</TARGET> ja olen tyytyväinen.\tpositive\t1.0\tpositive,positive,positive,positive,positive\n",
            "comments2003a-590164-4.1\t<TARGET>The Beatles</TARGET> on edelleenki maailma parasta musiikkia, eikä sitä voita mikään!\tpositive\t1.0\tpositive,positive,positive,positive,positive\n",
            "threads2011b-327878-1.2\tRoskaruokaa mukaan.. ... myyville (Mäcit, <TARGET>Heset</TARGET> ymv.) pitäisi laittaa jäte-/roskaamismaksu.\tnegative\t0.6666666666666666\tnegative,positive,negative,negative,negative,neither\n",
            "comments2014d-822316-1.1\tJos jotain pitäisi alkaa tutkia, on se <TARGET>rfsu:n</TARGET> motiivit ilmoittaa julkisesti selvästi todellisuutta pienempiä lukuja.\tnegative\t1.0\tnegative,negative,negative\n",
            "comments2005c-69174-2.2\tVene ei ole Wellu, ihan <TARGET>Yamarin</TARGET> -merkkinen vaan, mutta kohtuu nopea kuitenkin (n.38 s.)\tpositive\t0.75\tpositive,neither,positive,positive\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We see above that the first line is a \"header\" starting with the `#` character used for comments in Python code and the lines contain five fields separated by TAB characters:\n",
        "\n",
        "* `ID`: unique identifier (not required here)\n",
        "* `text`: the text of the example, with sentiment target marked with `<TARGET>...</TARGET>`\n",
        "* `majority`: the label assigned to the example by the majority of annotators\n",
        "* `agreement`: the ratio of all annotators that assigned the majority label\n",
        "* `annotations`: a comma-separated list of all of the annotations for the example"
      ],
      "metadata": {
        "id": "OIU7iAb_0ZBF"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Reformat data for `datasets` package\n",
        "\n",
        "* You really only need to do this once\n",
        "* Make a [*json lines*](https://en.wikipedia.org/wiki/JSON_streaming#Line-delimited_JSON) (JSONL) file with one [JSON](https://en.wikipedia.org/wiki/JSON)-encoded example per line\n",
        "* Each example has the `text` and the `label` as an integer\n",
        "* We have four different labels in this particular data"
      ],
      "metadata": {
        "id": "ygsR7Wr_0kIz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import re\n",
        "import json\n",
        "\n",
        "label_names=[\"positive\",\"negative\",\"reject\",\"neither\"]\n",
        "data=[]\n",
        "with open(\"sentiment-target-fi.tsv\") as f:\n",
        "    for line in f:\n",
        "        line=line.rstrip(\"\\n\")\n",
        "        if not line or line.startswith(\"#\"): #skip empty and comments (incl. header)\n",
        "            continue\n",
        "        cols=line.split(\"\\t\")\n",
        "        if len(cols)!=5: #skip weird lines that don't have the right number of columns\n",
        "            continue\n",
        "        data.append(cols)"
      ],
      "metadata": {
        "id": "nH0NWm500rxs"
      },
      "execution_count": 73,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We'll randomly shuffle the data to assure that our split into training, validation and test data is random.\n",
        "\n",
        "Note that here we call [`random.seed`](https://docs.python.org/3/library/random.html#bookkeeping-functions) with an arbitrary integer value so that the shuffle always results in the same split. This is not necessary (and must be avoided if you e.g. need different shuffles!), but useful here as it will make results in this notebook more stable. "
      ],
      "metadata": {
        "id": "zPm8DeY-0wpI"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import random\n",
        "\n",
        "random.seed(1234)    # seed with arbitrary integer value\n",
        "random.shuffle(data) # produces arbitrary but fixed order"
      ],
      "metadata": {
        "id": "wZPI5YqV01M5"
      },
      "execution_count": 74,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Now, let's cast the data into simple python dictionaries and save these as JSONL."
      ],
      "metadata": {
        "id": "Ky4OSwuC04ob"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "with open(\"sentiment-data.jsonl\", \"wt\") as f:\n",
        "    for cols in data:\n",
        "        item = {\n",
        "            \"text\": cols[1],\n",
        "            \"label\": label_names.index(cols[2]),    # translate from label strings to integers\n",
        "        }\n",
        "        print(json.dumps(item,ensure_ascii=False,sort_keys=True),file=f)"
      ],
      "metadata": {
        "id": "uO3JAsCM07ka"
      },
      "execution_count": 75,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's read back the file we just created and print one example to make sure it looks as intended:"
      ],
      "metadata": {
        "id": "xBbAC1vK4h73"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "examples = []\n",
        "with open(\"sentiment-data.jsonl\") as f:\n",
        "    for line in f:\n",
        "        example = json.loads(line)\n",
        "        examples.append(example)\n",
        "\n",
        "pprint(examples[0])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "lKBayVyC4mZz",
        "outputId": "8e000ff2-aeee-42e5-d2e0-fafe9ac13a77"
      },
      "execution_count": 76,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'label': 3,\n",
            " 'text': '<TARGET>BKA:n</TARGET> tilastot koskevat vuoden 2016 ensimmäistä '\n",
            "         'yhdeksää kuukautta.'}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Load data with `datasets`\n",
        "\n",
        "As you know, the `datasets` library provides broad support for [loading datasets](https://huggingface.co/docs/datasets/loading_datasets.html) through the `load_dataset` function, which we will use here to load the JSONL-formatted data we prepared just above."
      ],
      "metadata": {
        "id": "peXvwikS46aX"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import datasets\n",
        "\n",
        "file_name = \"sentiment-data.jsonl\"\n",
        "dataset = datasets.load_dataset(\n",
        "    'json',                             # Format of the data\n",
        "    data_files={\"everything\":file_name},    # All data files, here we only have one\n",
        "    split={\n",
        "        \"train\":\"everything[:80%]\",  # First 80% is the train set\n",
        "        \"validation\":\"everything[80%:90%]\",   # Next 10% is the validation/dev set\n",
        "        \"test\":\"everything[90%:]\"    # last 10% is the test set\n",
        "    },\n",
        "    features=datasets.Features({    # Here we tell how to interpret the attributes\n",
        "        \"label\":datasets.ClassLabel(names=label_names),\n",
        "        \"text\":datasets.Value(\"string\")\n",
        "    })\n",
        ")"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 189,
          "referenced_widgets": [
            "50d593ed27aa4975bbbe1be40f847361",
            "e6c35ef27050487d893ea52b0d246fac",
            "7497f66b64a64ef89ed4a0c4900a1bf0",
            "e7bdc50578764db0a39eee44cae7fffa",
            "13d8146a1a0643b9803172d718c13599",
            "30f6173dac894bcfa8365b15e63b0083",
            "e30d5391d3664fb39728e1eb389289f8",
            "f131fdfefcd646b0aa617d987dc90b75",
            "fa1b41850b5b422f9de598e687dba64b",
            "2057f4bf3a90494088dab00e16f4c3c5",
            "cdd380742b1043ad9ba02c09ef29c197",
            "434964fddb0644f8800559aa4a4313cd",
            "3f45f6f7c34c4887bcdca515df276271",
            "069a38cd37e04b64be5d1ad4b59f26b5",
            "99eaa73e2fda48a4991db78640469ba4",
            "31ef362c75ee4c5889ef2cbb70382531",
            "e6d51fdbfcc247938a5b1aae4ed1162f",
            "2fb9fe33de8d4d16897d25abcda2e562",
            "d2c1b9dd53f1427b9723fef8a5cb1c3f",
            "133fdb0e0125418eb0be15c912a5e40d",
            "7eaa16fb81ba4896a1592538af246d21",
            "b18c388639cd4010a0e8020b8bf6a67c",
            "f3dde39fddaf450a98df72c3cc9bc517",
            "3634fb01fbfa4145b2e848e643354cc9",
            "a8049da7abf743d9b8cb78785ffabdf1",
            "e03ee90340954b0bacc52e763f75f822",
            "5e755e85ff1040c9bd0b0ae2fe226894",
            "dc8516a00cb9439eb28123ec24029524",
            "afbaa639e9ce4075aa785270ef4fcd99",
            "10e47ff402154ba9801330de0d72f537",
            "fe876839751545a6a593832de77e5299",
            "f809c868c6a449f5ba36ff5808277659",
            "df4c2b0ea4e2435ebc47f98951bf20d8"
          ]
        },
        "id": "5DKskTuoyCf-",
        "outputId": "d1f0bdd6-2498-4c82-99bc-5f4c915b6fdd"
      },
      "execution_count": 77,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Using custom data configuration default-f648b673909eb59e\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Downloading and preparing dataset json/default to /root/.cache/huggingface/datasets/json/default-f648b673909eb59e/0.0.0/ac0ca5f5289a6cf108e706efcf040422dbbfa8e658dee6a819f20d76bb84d26b...\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading data files:   0%|          | 0/1 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "50d593ed27aa4975bbbe1be40f847361"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Extracting data files:   0%|          | 0/1 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "434964fddb0644f8800559aa4a4313cd"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Dataset json downloaded and prepared to /root/.cache/huggingface/datasets/json/default-f648b673909eb59e/0.0.0/ac0ca5f5289a6cf108e706efcf040422dbbfa8e658dee6a819f20d76bb84d26b. Subsequent calls will reuse this data.\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/3 [00:00<?, ?it/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "f3dde39fddaf450a98df72c3cc9bc517"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's see what the dataset contains:"
      ],
      "metadata": {
        "id": "FVo6_CS2eeIq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(dataset)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "y3K4Fkz7yce7",
        "outputId": "d4cd2f1b-6f44-4a92-b834-c29911d32402"
      },
      "execution_count": 78,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "DatasetDict({\n",
            "    train: Dataset({\n",
            "        features: ['label', 'text'],\n",
            "        num_rows: 2743\n",
            "    })\n",
            "    validation: Dataset({\n",
            "        features: ['label', 'text'],\n",
            "        num_rows: 343\n",
            "    })\n",
            "    test: Dataset({\n",
            "        features: ['label', 'text'],\n",
            "        num_rows: 343\n",
            "    })\n",
            "})\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "and print out an example:"
      ],
      "metadata": {
        "id": "o17ZWi_kf4AJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "pprint(dataset[\"train\"][0])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Wd__TIUnf9PD",
        "outputId": "03c05c1b-c398-46d1-a035-31e358c656e5"
      },
      "execution_count": 79,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'label': 3,\n",
            " 'text': '<TARGET>BKA:n</TARGET> tilastot koskevat vuoden 2016 ensimmäistä '\n",
            "         'yhdeksää kuukautta.'}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "---\n",
        "\n",
        "# Tokenize and vectorize data\n",
        "\n",
        "(This part of the code follows previous notebooks that you should already be familiar with.)"
      ],
      "metadata": {
        "id": "8KF9UtzUbrBA"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "To tokenize and vectorize the texts of our dataset, we will again use previously created tokenizers through the simple [`AutoTokenizer`](https://huggingface.co/docs/transformers/model_doc/auto#transformers.AutoTokenizer) class.\n",
        "\n",
        "The [`AutoTokenizer.from_pretrained`](https://huggingface.co/docs/transformers/model_doc/auto#transformers.AutoTokenizer.from_pretrained) function can load the tokenizer associated with any of the large number of models found in the [Hugging Face models repository](https://huggingface.co/models). Here, our texts are in Finnish, and we'll load the tokenizer for the [`TurkuNLP/bert-base-finnish-cased-v1`](https://huggingface.co/TurkuNLP/bert-base-finnish-cased-v1) model.\n"
      ],
      "metadata": {
        "id": "H9RBff6Rkfyt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import transformers\n",
        "\n",
        "model_name = \"TurkuNLP/bert-base-finnish-cased-v1\"\n",
        "tokenizer = transformers.AutoTokenizer.from_pretrained(model_name)"
      ],
      "metadata": {
        "id": "wjrAGcFtymJF"
      },
      "execution_count": 80,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "As in the [MLP notebook](https://github.com/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/hf_trainer_mlp.ipynb), we will define a simple tokenization function and tokenize and vectorize our whole dataset with the tokenizer by calling the [`Dataset.map`](https://huggingface.co/docs/datasets/package_reference/main_classes.html#datasets.Dataset.map) function.\n",
        "\n",
        "Note that here we're providing a `max_length` argument and `truncation=True` in the tokenizer call. This limits the maximum length of outputs to the given length (see the [tokenizers documentation](https://huggingface.co/docs/transformers/preprocessing#everything-you-always-wanted-to-know-about-padding-and-truncation) for details). **The maximum sequence length for BERT models is usually 512 due to the trained position embeddings.**"
      ],
      "metadata": {
        "id": "mjEeyPsXc12X"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Define a simple function that applies the tokenizer\n",
        "\n",
        "# maximum length of BERT models is 512 due to the position embeddings\n",
        "def tokenize(example):\n",
        "    return tokenizer(\n",
        "        example[\"text\"],\n",
        "        max_length=512,\n",
        "        truncation=True,\n",
        "    )\n",
        "\n",
        "# Apply the tokenizer to the whole dataset using .map()\n",
        "dataset = dataset.map(tokenize)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 113,
          "referenced_widgets": [
            "1ad9b0850ab94a938b61de6bed990482",
            "da09dde448ee49b18c34eb2fb6e2a2f2",
            "40d1a7eeac9c4fbbb14ea151c01a2357",
            "4fe3f1b32997464080d6456a66dd2cb5",
            "291af4a7e0254ccea17b9fa0827ff300",
            "b359af007c8849d4a8a0187bedd4ab2c",
            "16e48341d7724f4f927b978c745108a5",
            "6da1d8566ac441cdadbb8b211b9b164a",
            "cc70cfe677124a4d9df99c0fc327ff44",
            "de7237cc0a6b47fcbc3ae22b6db02b8b",
            "10f81f2778c84d918796329d855b99a0",
            "4593b4744ab94b20bf988c06dc926628",
            "15c90f3f94e346a3be88e7c1a0a2da58",
            "05967551ce224aa4b11b197364bb71e8",
            "bca951e1f3194ff19b2885238e0e453d",
            "d96c9cf6d49f431c9fbeaf5fa342b138",
            "d7ecd06b99d2435283f0751314e3146a",
            "80419402cd0e4a9b9449a71a9a25adfb",
            "41e09c4f36f749e6b27ab9d14f4ce37f",
            "2748a0b464cf4a48a7e2cb8504b34b46",
            "bd05e0ba4aa343529c57559358c4dde6",
            "bfbac8f75b7846629fb4d0e34d2756bb",
            "57f73569d7bf43caaa746bda9df0af42",
            "8f885c7148bd4df3a4b0bce9e4078bf2",
            "c10c70680c6346d5bc7699b9022a122a",
            "0f642d39819c4500a7e9700b6abbf89d",
            "294743a2749e4eccb5eecdff89682d9e",
            "087901849abb4ecd8d2fbe52b3cec88d",
            "792fa0940156469bb2b2154cdd9c51fa",
            "2101e89a3b2042aa85d719fa569b9681",
            "4957c647784d495083fc6df9094db54f",
            "447aeed254a04383bf739a1637619394",
            "fcb3a85c62b54ad89985c556ee83d7f5"
          ]
        },
        "id": "K2c4lFKTywlu",
        "outputId": "1a2be1fe-c40a-48c6-f523-8b5c7879678d"
      },
      "execution_count": 81,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/2743 [00:00<?, ?ex/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "1ad9b0850ab94a938b61de6bed990482"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/343 [00:00<?, ?ex/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "4593b4744ab94b20bf988c06dc926628"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/343 [00:00<?, ?ex/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "57f73569d7bf43caaa746bda9df0af42"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "---\n",
        "\n",
        "# Build model\n",
        "\n",
        "(This part of the code follows the [Text classification with BERT notebook](https://github.com/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/bert_text_classification.ipynb))"
      ],
      "metadata": {
        "id": "AiX7WeW2X5gv"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "* Ready-made model implementation from [HF transformers](https://huggingface.co/transformers/)\n",
        "* BertForSequenceClassification (sequence of tokens in, label out)\n",
        "  * Pre-trained BERT encoder\n",
        "  * Pooler output: CLS embedding + linear layer + tanh\n",
        "  * Classification layer\n",
        "* [Source code](https://github.com/huggingface/transformers/blob/v4.17.0/src/transformers/models/bert/modeling_bert.py#L1501)"
      ],
      "metadata": {
        "id": "GtGpvBKNFSeh"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "model = transformers.AutoModelForSequenceClassification.from_pretrained(\n",
        "    model_name,\n",
        "    num_labels=len(label_names)\n",
        ")"
      ],
      "metadata": {
        "id": "9kyQQhu0_wep",
        "outputId": "08e576c9-b429-4c18-e7bf-bb7e4881bcfb",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 82,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "Some weights of the model checkpoint at TurkuNLP/bert-base-finnish-cased-v1 were not used when initializing BertForSequenceClassification: ['cls.predictions.transform.dense.weight', 'cls.seq_relationship.bias', 'cls.predictions.transform.dense.bias', 'cls.predictions.decoder.weight', 'cls.seq_relationship.weight', 'cls.predictions.transform.LayerNorm.weight', 'cls.predictions.bias', 'cls.predictions.transform.LayerNorm.bias']\n",
            "- This IS expected if you are initializing BertForSequenceClassification from the checkpoint of a model trained on another task or with another architecture (e.g. initializing a BertForSequenceClassification model from a BertForPreTraining model).\n",
            "- This IS NOT expected if you are initializing BertForSequenceClassification from the checkpoint of a model that you expect to be exactly identical (initializing a BertForSequenceClassification model from a BertForSequenceClassification model).\n",
            "Some weights of BertForSequenceClassification were not initialized from the model checkpoint at TurkuNLP/bert-base-finnish-cased-v1 and are newly initialized: ['classifier.bias', 'classifier.weight']\n",
            "You should probably TRAIN this model on a down-stream task to be able to use it for predictions and inference.\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "---\n",
        "\n",
        "# Configure and train model\n",
        "\n",
        "(This part of the code follows the [Text classification with BERT notebook](https://github.com/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/bert_text_classification.ipynb))"
      ],
      "metadata": {
        "id": "Tmj91CdCT-fJ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "The following are hyperparameters that you may wish to adjust:\n",
        "\n",
        "* `learning_rate`: the step size for weight updates\n",
        "* `per_device_train_batch_size`: number of examples per training batch\n",
        "* `per_device_eval_batch_size`: number of examples per evaluate batch (larger number makes evaluation faster, but requires more memory)\n",
        "* `max_steps`: the maximum number of steps to train for"
      ],
      "metadata": {
        "id": "q5xkBXWDUL-_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Set training arguments\n",
        "trainer_args = transformers.TrainingArguments(\n",
        "    \"checkpoints\",\n",
        "    evaluation_strategy=\"steps\",\n",
        "    logging_strategy=\"steps\",\n",
        "    load_best_model_at_end=True,\n",
        "    eval_steps=100,\n",
        "    logging_steps=100,\n",
        "    learning_rate=0.00001,\n",
        "    per_device_train_batch_size=8,\n",
        "    per_device_eval_batch_size=32,\n",
        "    max_steps=500,\n",
        ")"
      ],
      "metadata": {
        "id": "gVW-yGhYItWk"
      },
      "execution_count": 83,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We'll then define the standard accuracy metric (ratio of correct out of all predictions), create a [DataCollatorWithPadding](https://huggingface.co/docs/transformers/main_classes/data_collator#transformers.DataCollatorWithPadding) to pad inputs to the same length (as required for batching) and an [EarlyStoppingCallback](https://huggingface.co/docs/transformers/main_classes/callback#transformers.EarlyStoppingCallback) to stop training when performance fails to improve for the given number of evaluations.\n",
        "\n",
        "(These should all be familiar to you from the [MLP notebook](https://github.com/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/hf_trainer_mlp.ipynb))"
      ],
      "metadata": {
        "id": "lRE6YtHvUTcj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "accuracy = datasets.load_metric(\"accuracy\")\n",
        "\n",
        "def compute_accuracy(outputs_and_labels):\n",
        "    outputs, labels = outputs_and_labels\n",
        "    predictions = outputs.argmax(axis=-1) #pick the index of the \"winning\" label\n",
        "    return accuracy.compute(predictions=predictions, references=labels)\n",
        "\n",
        "\n",
        "data_collator = transformers.DataCollatorWithPadding(tokenizer)\n",
        "\n",
        "# Argument gives the number of steps of patience before early stopping\n",
        "early_stopping = transformers.EarlyStoppingCallback(\n",
        "    early_stopping_patience=5\n",
        ")"
      ],
      "metadata": {
        "id": "miilVFfCIhR1"
      },
      "execution_count": 84,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Finally, as in the [CNN notebook](https://github.com/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/hf_trainer_cnn.ipynb), we'll create a simple custom [callback](https://huggingface.co/docs/transformers/main_classes/callback) to store values logged during training so that we can more easily examine them later. (This is only needed for visualization and is not necessary to understand in detail.)"
      ],
      "metadata": {
        "id": "pd3o8RIdaWqb"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from collections import defaultdict\n",
        "\n",
        "class LogSavingCallback(transformers.TrainerCallback):\n",
        "    def on_train_begin(self, *args, **kwargs):\n",
        "        self.logs = defaultdict(list)\n",
        "        self.training = True\n",
        "\n",
        "    def on_train_end(self, *args, **kwargs):\n",
        "        self.training = False\n",
        "\n",
        "    def on_log(self, args, state, control, logs, model=None, **kwargs):\n",
        "        if self.training:\n",
        "            for k, v in logs.items():\n",
        "                if k != \"epoch\" or v not in self.logs[k]:\n",
        "                    self.logs[k].append(v)\n",
        "\n",
        "training_logs = LogSavingCallback()"
      ],
      "metadata": {
        "id": "0EFQebkDYXES"
      },
      "execution_count": 85,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We then pass the model, trainer arguments, training and evaluation data, metric, the collator, and the callbacks to a [Trainer](https://huggingface.co/docs/transformers/main_classes/trainer) and call `.train()` to train the model."
      ],
      "metadata": {
        "id": "GCuvqB0bVPI2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "trainer = transformers.Trainer(\n",
        "    model=model,\n",
        "    args=trainer_args,\n",
        "    train_dataset=dataset[\"train\"],\n",
        "    eval_dataset=dataset[\"validation\"],\n",
        "    compute_metrics=compute_accuracy,\n",
        "    data_collator=data_collator,\n",
        "    tokenizer = tokenizer,\n",
        "    callbacks=[early_stopping, training_logs]\n",
        ")\n",
        "\n",
        "trainer.train()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 308
        },
        "id": "4lKiSRNJVOvJ",
        "outputId": "e23f57c0-8630-447b-a112-a3674502d217"
      },
      "execution_count": 86,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/transformers/optimization.py:309: FutureWarning: This implementation of AdamW is deprecated and will be removed in a future version. Use the PyTorch implementation torch.optim.AdamW instead, or set `no_deprecation_warning=True` to disable this warning\n",
            "  FutureWarning,\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='500' max='500' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [500/500 02:34, Epoch 1/2]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              " <tr style=\"text-align: left;\">\n",
              "      <th>Step</th>\n",
              "      <th>Training Loss</th>\n",
              "      <th>Validation Loss</th>\n",
              "      <th>Accuracy</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>100</td>\n",
              "      <td>0.957200</td>\n",
              "      <td>0.704318</td>\n",
              "      <td>0.766764</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>200</td>\n",
              "      <td>0.598500</td>\n",
              "      <td>0.608934</td>\n",
              "      <td>0.778426</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>300</td>\n",
              "      <td>0.661600</td>\n",
              "      <td>0.615153</td>\n",
              "      <td>0.775510</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>400</td>\n",
              "      <td>0.541300</td>\n",
              "      <td>0.537914</td>\n",
              "      <td>0.822157</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>500</td>\n",
              "      <td>0.467300</td>\n",
              "      <td>0.521078</td>\n",
              "      <td>0.825073</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TrainOutput(global_step=500, training_loss=0.6451809387207031, metrics={'train_runtime': 154.9258, 'train_samples_per_second': 25.819, 'train_steps_per_second': 3.227, 'total_flos': 97220252949456.0, 'train_loss': 0.6451809387207031, 'epoch': 1.46})"
            ]
          },
          "metadata": {},
          "execution_count": 86
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "---\n",
        "\n",
        "# Results"
      ],
      "metadata": {
        "id": "VId9dMvAO6aA"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Evaluate and print out results:"
      ],
      "metadata": {
        "id": "h0VrtFCrO-LO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "eval_results = trainer.evaluate(dataset[\"test\"])\n",
        "\n",
        "pprint(eval_results)\n",
        "\n",
        "print('Accuracy:', eval_results['eval_accuracy'])"
      ],
      "metadata": {
        "id": "7zw3iJ7tPS7V",
        "outputId": "03cb82b2-eb8d-4105-bf9f-8d9a83411ccf",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 167
        }
      },
      "execution_count": 87,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='11' max='11' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [11/11 00:02]\n",
              "    </div>\n",
              "    "
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'epoch': 1.46,\n",
            " 'eval_accuracy': 0.8396501457725948,\n",
            " 'eval_loss': 0.48228707909584045,\n",
            " 'eval_runtime': 2.448,\n",
            " 'eval_samples_per_second': 140.112,\n",
            " 'eval_steps_per_second': 4.493}\n",
            "Accuracy: 0.8396501457725948\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's also have a look at training and evaluation loss and evaluation accuracy progression as we did in the [CNN notebook](https://github.com/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/hf_trainer_cnn.ipynb). (The code here is only for visualization and you do not need to understand it, but you should aim to be able to interpret the plots.)"
      ],
      "metadata": {
        "id": "xm1ZAFTObyf8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%matplotlib inline\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "def plot(logs, keys, labels):\n",
        "    values = sum([logs[k] for k in keys], [])\n",
        "    plt.ylim(max(min(values)-0.1, 0.0), min(max(values)+0.1, 1.0))\n",
        "    for key, label in zip(keys, labels):    \n",
        "        plt.plot(logs[\"epoch\"], logs[key], label=label)\n",
        "    plt.legend()\n",
        "    plt.show()\n",
        "\n",
        "plot(training_logs.logs, [\"loss\", \"eval_loss\"], [\"Training loss\", \"Evaluation loss\"])"
      ],
      "metadata": {
        "id": "4C2fPl02U9kq",
        "outputId": "53c09e5e-2dbc-4546-da1b-2ffe27c133bf",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 269
        }
      },
      "execution_count": 88,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXQAAAD8CAYAAABn919SAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nO3deVyU5f7/8dfFIsgiKpss7rKogKC4pKWouaSmaZvtVifb7ZstVqfFOqdvnU7b6WTHlm+n9Xeqc06ahWmZ4pKWYpqCAqKioogIsiiyzVy/P+6RSFEGGGZg+DwfDx7O3HPPPZ9b6801131f16W01gghhGj7XBxdgBBCCNuQQBdCCCchgS6EEE5CAl0IIZyEBLoQQjgJCXQhhHASDQa6Uup9pdQxpVTaeV5XSqk3lFLZSqkdSqnBti9TCCFEQ6xpoX8ATL7A65cBEZafucA/ml+WEEKIxmow0LXW64CiC+wyA/hIG34COiulQmxVoBBCCOu42eAYYcChOs9zLdvyzt5RKTUXoxWPt7f3kOjoaBt8vBBCtB9bt249rrUOrO81WwS61bTW7wDvACQmJurU1FR7frwQQrR5SqkD53vNFne5HAa613kebtkmhBDCjmwR6MuAmy13u4wASrTW53S3CCGEaFkNdrkopf4FJAEBSqlc4BnAHUBrvRhYDkwBsoFy4NaWKlYIIcT5NRjoWuvrGnhdA/farCIhhE1VV1eTm5tLRUWFo0sRjeDp6Ul4eDju7u5Wv8euF0WFEPaXm5uLr68vvXr1Qinl6HKEFbTWFBYWkpubS+/eva1+nwz9F8LJVVRU4O/vL2Hehiil8Pf3b/S3Kgl0IdoBCfO2pyn/ZhLoQgjhJCTQhRAtqrCwkPj4eOLj4+nWrRthYWG1z6uqqi743tTUVObNm9fgZ4wcOdImtaakpDBt2jSbHMsR5KKoEKJF+fv7s337dgAWLlyIj48PDz/8cO3rNTU1uLnVH0WJiYkkJiY2+BkbN260TbFtnLTQhRB2N2fOHO666y6GDx/Oo48+yubNm7noootISEhg5MiRZGZmAr9vMS9cuJDbbruNpKQk+vTpwxtvvFF7PB8fn9r9k5KSuOqqq4iOjuaGG27AuLMali9fTnR0NEOGDGHevHkNtsSLioq44ooriIuLY8SIEezYsQOAtWvX1n7DSEhIoKysjLy8PEaPHk18fDwxMTGsX7/e5n9n1pAWuhDtyLNfp7PrSKlNjzkgtBPPXD6w0e/Lzc1l48aNuLq6Ulpayvr163Fzc2PVqlU88cQT/Pe//z3nPRkZGaxZs4aysjKioqK4++67z7lPe9u2baSnpxMaGsqoUaP48ccfSUxM5M4772TdunX07t2b66674PAaAJ555hkSEhJYunQpq1ev5uabb2b79u28/PLLLFq0iFGjRnHy5Ek8PT155513mDRpEn/84x8xmUyUl5c3+u/DFtpcC311Rj63fbCFGpPZ0aUIIZrh6quvxtXVFYCSkhKuvvpqYmJiePDBB0lPT6/3PVOnTsXDw4OAgACCgoLIz88/Z59hw4YRHh6Oi4sL8fHx5OTkkJGRQZ8+fWrv6bYm0Dds2MBNN90EwLhx4ygsLKS0tJRRo0Yxf/583njjDYqLi3Fzc2Po0KH885//ZOHChezcuRNfX9+m/rU0S5troVdWm1mdcYwPNx3g9outv+FeCEGTWtItxdvbu/bxU089xdixY1myZAk5OTkkJSXV+x4PD4/ax66urtTU1DRpn+Z47LHHmDp1KsuXL2fUqFGsXLmS0aNHs27dOpKTk5kzZw7z58/n5ptvtunnWqPNtdAnx3RjTGQgr36XydESGcoshDMoKSkhLCwMgA8++MDmx4+KimLfvn3k5OQA8Pnnnzf4nksuuYRPP/0UMPrmAwIC6NSpE3v37iU2NpYFCxYwdOhQMjIyOHDgAMHBwdxxxx384Q9/4JdffrH5OVijzQW6UornZgykxqz5U/IuR5cjhLCBRx99lMcff5yEhASbt6gBOnbsyFtvvcXkyZMZMmQIvr6++Pn5XfA9CxcuZOvWrcTFxfHYY4/x4YcfAvD6668TExNDXFwc7u7uXHbZZaSkpDBo0CASEhL4/PPPeeCBB2x+DtZQZ64A21tzF7h444c9vPp9Fh/dNozRkfUu3iGEAHbv3k3//v0dXYbDnTx5Eh8fH7TW3HvvvURERPDggw86uqwLqu/fTim1VWtd772cba6FfsadY/rQO8Cbp79Ko6La5OhyhBCt3Lvvvkt8fDwDBw6kpKSEO++809El2VybDXQPN1f+NCOGnMJyFq/d6+hyhBCt3IMPPsj27dvZtWsXn376KV5eXo4uyebabKADXBwRwOWDQnkrZS85x085uhwhhHCoNh3oAE9O7U8HVxeeXpaOo64HCCFEa9DmAz24kycPTYxkXVYBy3cedXQ5QgjhMG0+0AFuGtGTgaGdeO6bdMoqqh1djhBCOIRTBLqbqwt/viKGY2WVvL5qj6PLEUKcxdXVtXZCq/j4eF588cUmHScpKYmm3u6ckpLyu1kZFy9ezEcffdSkY9WVk5NDTExMs49jC21u6P/5JPTownXDevDBxhyuHBzOgNBOji5JCGHRsWPH2il0HSUlJQUfH5/audPvuusuh9bTEpyihX7GgknRdO7ozpNLd2I2ywVSIVqzFStWcPXVV9c+rztV7t13301iYiIDBw7kmWeeqff9Z6bMBfjPf/7DnDlzAPj6668ZPnw4CQkJXHrppeTn55OTk8PixYt57bXXiI+PZ/369SxcuJCXX34ZgO3btzNixAji4uKYOXMmJ06cAIxvBAsWLGDYsGFERkY2OC1uRUUFt956K7GxsSQkJLBmzRoA0tPTGTZsGPHx8cTFxbFnzx5OnTrF1KlTGTRoEDExMVZNR9AQp2mhA/h5ufP4lP48/O9f+SL1ELOH9XB0SUK0Lt8+Bkd32vaY3WLhsgt3oZw+fZr4+Pja548//jhXXnklc+fO5dSpU3h7e/P5558ze/ZsAJ5//nm6du2KyWRi/Pjx7Nixg7i4OKvKufjii/npp59QSvHee+/x0ksv8corr3DXXXf9bnGNH374ofY9N998M3//+98ZM2YMTz/9NM8++yyvv/46YCzAsXnzZpYvX86zzz7LqlWrzvvZixYtQinFzp07ycjIYOLEiWRlZbF48WIeeOABbrjhBqqqqjCZTCxfvpzQ0FCSk5MBYz6b5nKqFjrAlYPDGNa7Ky+uyKDo1IWXtxJC2MeZLpczP9deey1ubm5MnjyZr7/+mpqaGpKTk5kxYwYAX3zxBYMHDyYhIYH09HR27bJ+3qbc3FwmTZpEbGwsf/3rX887Fe8ZJSUlFBcXM2bMGABuueUW1q1bV/v6rFmzABgyZEjt5F7ns2HDBm688UYAoqOj6dmzJ1lZWVx00UX87//+L3/5y184cOAAHTt2JDY2lu+//54FCxawfv36BueWsYZTtdDBmLzrz1fEMOVv63lh+W7+evUgR5ckROvRQEva3mbPns2bb75J165dSUxMxNfXl/379/Pyyy+zZcsWunTpwpw5c6ioOHdmVaVU7eO6r99///3Mnz+f6dOnk5KSwsKFC5tV45npeJszFe/111/P8OHDSU5OZsqUKbz99tuMGzeOX375heXLl/Pkk08yfvx4nn766WbV6nQtdIDIYF9uv6Q3/96ay5acIkeXI4Q4jzFjxvDLL7/w7rvv1na3lJaW4u3tjZ+fH/n5+Xz77bf1vjc4OJjdu3djNptZsmRJ7fa6U/GemSERwNfXl7KysnOO4+fnR5cuXWr7xz/++OPa1npj1Z1yNysri4MHD9ZO3dunTx/mzZvHjBkz2LFjB0eOHMHLy4sbb7yRRx55xCZT7loV6EqpyUqpTKVUtlLqsXpe76mU+kEptUMplaKUCm92Zc30wPgIwjp35MklaVTL6kZCONSZPvQzP489ZsSIq6sr06ZN49tvv629IHpmGtro6Giuv/56Ro0aVe8xX3zxRaZNm8bIkSMJCQmp3b5w4UKuvvpqhgwZQkBAQO32yy+/nCVLltReFK3rww8/5JFHHiEuLo7t27c3uaV8zz33YDabiY2N5dprr+WDDz7Aw8ODL774gpiYGOLj40lLS+Pmm29m586dtRdKn332WZ588skmfWZdDU6fq5RyBbKACUAusAW4Tmu9q84+/wa+0Vp/qJQaB9yqtb7pQsdt7vS51vgu/ShzP97KE1OimTu6b4t+lhCtlUyf23a1xPS5w4BsrfU+rXUV8Bkw46x9BgCrLY/X1PO6Q0wYEMz46CBeX7WHI8WnHV2OEEK0KGsCPQw4VOd5rmVbXb8CsyyPZwK+Sin/sw+klJqrlEpVSqUWFBQ0pd5GUUqxcPpAzFrz3NeyupEQwrnZ6qLow8AYpdQ2YAxwGDhn1Qmt9Tta60StdWJgoH1WGere1Yv7x0WwIv0oazKO2eUzhWhtZCbStqcp/2bWBPphoHud5+GWbXU/+IjWepbWOgH4o2VbcaOraSF3XNKHvoHePL1MVjcS7Y+npyeFhYUS6m2I1prCwkI8PT0b9T5r7kPfAkQopXpjBPls4Pq6OyilAoAirbUZeBx4v1FVtLAObi786YoYrn/3ZxatyeahiVGOLkkIuwkPDyc3Nxd7dHMK2/H09CQ8vHE3DDYY6FrrGqXUfcBKwBV4X2udrpR6DkjVWi8DkoAXlFIaWAfc29jiW9rIvgHMTAhj8dq9XJEQRt9An4bfJIQTcHd3p3fv3o4uQ9hBg7ctthR73LZ4toKySsa9kkJcuB+f3D78dyPNhBCiLWjubYtOI9DXg0cnRfFjdiHLfj3i6HKEEMKm2lWgA1w/vCdx4X78OXk3pbK6kRDCibS7QHd1MSbvOn6ykle/y3J0OUIIYTPtLtAB4sI7c9OInny0KYe0w82fg1gIIVqDdhnoAA9NjKKrtwd/XLITk6xuJIRwAu020P06uvPk1P78mlvCvzYfdHQ5QgjRbO020AFmxIcysq8/L63IoKCs0tHlCCFEs7TrQFdK8dyMGE5Xm3hh+W5HlyOEEM3SrgMdoF+QD3NH9+HLbYf5aV+ho8sRQogma/eBDnDf2AjCu3TkyaVpVNXI6kZCiLZJAh3o2MGVZ6cPJPvYSf5vw35HlyOEEE0igW4xvn8wEwcE88YPe8g9Ue7ocoQQotEk0Ot4ZvpAABYuk9WNhBBtjwR6HWGdO/I/l0awanc+3+/Kd3Q5QgjRKBLoZ7nt4t5EBvuwcFk65VU1ji5HCCGsJoF+FndXF/58RSyHi0/z99XZji5HCCGsJoFej2G9u3LVkHDeXbePPfllji5HCCGsIoF+Ho9fFo23hxtPfZUmi+sKIdoECfTz8PfxYMHkaH7aV8SSbYcdXY4QQjRIAv0CZg/tTnz3zvzv8t2UlMvqRkKI1k0C/QJcLKsbFZ2q4q/fZTi6HCGEuCAJ9AbEhPlxy8hefPrzQbYfKnZ0OUIIcV4S6FaYPyGSQB8PnlwqqxsJIVovCXQr+Hq689S0AaQdLuWTnw44uhwhhKiXBLqVpsWFcElEAC+vzORYaYWjyxFCiHNYFehKqclKqUylVLZS6rF6Xu+hlFqjlNqmlNqhlJpi+1Id68zqRpU1Zp6X1Y2EEK1Qg4GulHIFFgGXAQOA65RSA87a7UngC611AjAbeMvWhbYGvQO8uSupL19tP8KP2ccdXY4QQvyONS30YUC21nqf1roK+AyYcdY+GuhkeewHHLFdia3LPUl96envxVNL06isMTm6HCGEqGVNoIcBh+o8z7Vsq2shcKNSKhdYDtxf34GUUnOVUqlKqdSCgoImlOt4nu7G6kb7jp/i3XX7HF2OEELUstVF0euAD7TW4cAU4GOl1DnH1lq/o7VO1FonBgYG2uij7S8pKogpsd34++psDhbK6kZCiNbBmkA/DHSv8zzcsq2u24EvALTWmwBPIMAWBbZWT08biJuL4pllMnmXEKJ1sCbQtwARSqneSqkOGBc9l521z0FgPIBSqj9GoLfNPhUrdfPz5MEJkazJLGBluqxuJIRwvAYDXWtdA9wHrAR2Y9zNkq6Uek4pNd2y20PAHUqpX4F/AXN0O2i2zhnZi+huvjz7dTqnKmV1IyGEYylH5W5iYqJOTU11yGfb0tYDRVz5j03MHd2HJ6b0d3Q5Qggnp5TaqrVOrO81GSnaTEN6dmX20O7834b9ZB6V1Y2EEI4jgW4DCyZH08nTjSeX7sQsk3cJIRxEAt0Gunh34PHL+rMl5wT/+SXX0eUIIdopCXQbuWpIOIk9u/DC8t2cOFXl6HKEEO2QBLqNuLgo/nRFDKUVNby0UlY3EkLYnwS6DfUP6cRto3rxr82H2HrghKPLEUK0MxLoNvY/l0bSrZMnTy5No8ZkdnQ5Qoh2RALdxrw93Hjm8gHszivlw02yupEQwn4k0FvA5JhuJEUF8up3mRwtkdWNhBD2IYHeApRSPDc9hhqz5k/JuxxdjhCinZBAbyE9/L24d2w/knfksS7LqecpE0K0EhLoLejOMX3oE+DN01+lUVEtqxsJIVqWBHoL8nBz5bkZMeQUlrN47V5HlyOEcHIS6C3s4ogALh8Uylspe8k5fsrR5QghnJgEuh08NbU/Hq4uPPWVrG4khGg5Euh2ENTJk4cmRrJ+z3GW7zzq6HKEEE5KAt1ObhzRk4GhnXjum3TKKqodXY4QwglJoNuJm6sLz8+M5VhZJa99v8fR5QghnJAEuh3Fd+/M9cN68MHG/aQfKXF0OUIIJyOBbmePToqmi1cHnlqaJqsbCSFsSgLdzvy83HliSn9+OVjM56mHHF2OEMKJSKA7wKzBYQzr3ZUXv82g8GSlo8sRQjgJCXQHUErx5ytiOFVZw4vfyupGQgjbkEB3kMhgX/5wSR/+vTWXLTlFji5HCOEEJNAdaN74foR17siTS9KoltWNhBDNZFWgK6UmK6UylVLZSqnH6nn9NaXUdstPllKq2PalWjjR0HmvDsbqRpn5Zfzzx/2OLkcI0cY1GOhKKVdgEXAZMAC4Tik1oO4+WusHtdbxWut44O/Aly1RLAC7v4Z/TjX+NLf9KWknDuzGpf2DeH3VHo4Un3Z0OUKINsyaFvowIFtrvU9rXQV8Bsy4wP7XAf+yRXH10iYoPgif3whvxMPGN+F0y30hsIdnLh+IWWue+1pWNxJCNJ01gR4G1L1hOtey7RxKqZ5Ab2D1eV6fq5RKVUqlFhQ0cRWfgTNh3ja45mPw6w7f/RFeHQDJD8Px7KYd08G6d/Xi/nERrEg/ypqMY44uRwjRRtn6ouhs4D9a63r7QrTW72itE7XWiYGBgU3/FFc3GDAdbl0Od66DATPglw/hzSHwyVWQ/UOb62u/45I+9Avy4ellaZyuavtdSUII+7Mm0A8D3es8D7dsq89sWrK7pT4hg2DmP+DBdEh6AvJ+hU9mwaLhsOX/oKptLCrRwc2FP82I4VDRad5KaZvfNIQQjmVNoG8BIpRSvZVSHTBCe9nZOymlooEuwCbblmglnyBIWgAPpsHMt8G9IyTPN7pjvn8ailv/MPuL+vozMyGMxWv3srfgpKPLEUK0MQ0Guta6BrgPWAnsBr7QWqcrpZ5TSk2vs+ts4DPt6CV53Dxg0GyYmwK3rYQ+Y2Dj3+Fvg+CLm+HAplbdHfPElP54urvy1FJZ3UgI0TjKUaGRmJioU1NT7fNhxYdgy7uw9UOoKIaQeBhxt3GB1c3DPjU0wsebcnjqq3T+NjueGfH1Xn8WQrRTSqmtWuvE+l5rHyNFO3eHCc/B/F0w9VWoPg1L7oTXYiDlRTjZuu4suX54T+LC/fhz8m5KZXUjIYSV2kegn9HBG4beDvf+DDd+CaHxkPICvDYQltxtXFBtBVxdFM9fEcvxk5W8+l2Wo8sRQrQR7SvQz1AK+o2HG/4N96XC4Ftg11fw9mh4/zLjsanGoSXGhvtx04iefLQph7TDsrqREKJh7TPQ6wqIgKkvG90xE5+H0lzj4ukbCfDjG3D6hMNKe2hiFF29Pfjjkp2YZHUjIUQDJNDP6NgZRt4H87bDtZ9A5x7w/VPGbY/fzIcC+3d9+HV056lp/fk1t4R/bT5o988XQrQtEuhnc3GF/pfDrclw53oYOAu2fQKLhsInV8KeVWC231S30weFMrKvPy+tyKCgTFY3EkKcnwT6hYTEwRWLjFGoY/8IR3fCp1fComGw+V2obPnBP0opnpsRw+lqEy8s393in9eemcxa5qUXbVr7uA/dVmqqYNdS+OktOLINPP1g8M0wbK7RRdOCXl6ZyZtrsvls7ghG9PFv0c9qT4pOVbEuq4A1mcdYl1VAtUlzxyV9uP2S3vh4uDm6PCHOcaH70CXQm0JrOLQZfv4H7FoGaIieCiPugR4XGXfR2NjpKhMTXluLp7sry+ddQgc3+XLVFGazZufhElIyjRD/NbcYrcHfuwNjogI5WVHDd7vy8ffuwH3j+nH98B54uLk6umwhakmgt6SSXNjyHqT+0xiF2i3OGIUac6XNR6Guzsjntg9SeXRyFPck9bPpsZ1ZcXkV6/YcJyXjGGuzCig8VYVSEN+9M0mRQYyNDiQm1A8XF+MX8baDJ3hpRSab9hUS1rkjD06IZGZCGK4utv9FLURjSaDbQ1U57Pgcfl4MBRngHQiJt0Hi7eAbbLOPufPjVNZmFbBq/hjCu3jZ7LjOxGzW7MorJSXzGGsyC9h28ARmDV283BkTGUhSVBCjIwPp6t3hvMfQWrMh+zgvrchk5+ESIoJ8eHhSFBMHBKNa4BuYENaSQLcnrWHfGvhpMexZCS7uRmt9xF0QmtDswx8uPs2lr6xlVL8A3rul3n/TdqnkdDUb9hxnTabRCj9zR9CgcD/GRAUxNiqQuPDOjW5la635Nu0oL3+Xyb6CU8R378yjk6MY2TegJU5DiAZJoDtK4V74+W3Y/ilUnYTuI4zumOhpxiIdTfT22r288G0G796cyIQBtmv9tyVaa3bnlZGSdYyUjAK2HjyByazx6+jO6MhAkiIDGR0ZSKCvbbq9akxm/vtLLq+v2kNeSQWXRATw6KRoYsP9bHJ8Iawlge5oFSWw7VOjO6b4gLF03tA/GHfIeHVt9OGqTWamvrGeU5Umvp8/Gq8O7eNujLKKan7MPs6ajALWZhVwtLQCgIGhnRgbZfSFDwrvjJtry10wrqg28clPB1i0JpsT5dVMjQ1h/sRI+gb6tNhnClGXBHprYTZB1gr46R+Qsx7cvYy524ffBYFRjTrU5v1FXPP2Ju5O6suCydEtVLBjaa3Jyj9p6Qs/RmrOCWrMGl9PN0ZHBDImymiJB3XytHttpRXVvLduH+9t2E9ljZlrEsOZNz6CEL+Odq9FtC8S6K3R0TTjtscd/wZTJfQdB8Pvhn6Xgot1LcyH//0rS7cd5tsHLiEi2LeFC7aPU5U1/Jh9nJSsAlIyjnGkxGiFR3fzZWx0EGOjgkjo0Rn3FmyFN8bxk5W8uTqbT38+gFKKOSN7cfeYvnS5wAVXIZpDAr01O3Uctv7TWP+0LA/8+xkt9kHXgceFv8YXnqxk3Ctrie7my2dzR7TJuy+01uwtOFl7X/iW/SeoMpnx8XDj4n4BJEUZLfHW3vI9VFTO66v2sGRbLt4d3Jg7ug+3XdwbbxmcJGxMAr0tqKmC3cuMUaiHt4KHHwy+CYbdAV16nfdt/+/ngzyxZCevXjOIWYPD7VdvM5RX1bBpb2FtiOeeOA1AZLAPY6OCGBMVSGLPrm1y8FRWfhkvr8zku135BPh04L6x/bhOBicJG5JAb2sObbGMQv0KtBmiphh3x/Qcdc4oVLNZc+XijRwsLGf1Q0n4ebk7qOgL23/8FGsyjL7wn/cXUVVjxquDKyP7BjA22rg3PKxz626FN8YvB0/w0ooMftpXRHiXjsyfEMmMeBmcJJpPAr2tKjlsjELd+gGcLoLg2N9Gobr/diEw/UgJl/99A9cP78Gfr4h1XL11VFSb2LSvkLWWVviBwnIA+gZ6MzYqiKSoIIb27uLULVetNev3HOellRmkHS4lMtiHhydGMUEGJ4lmkEBv66pPw44vjLtjCnaDV4AxCnXo7eDbDYBnv07ng405LLlnFPHdOzukzIOF5azJPEZK5jE27i2kssaMp7uL0QqPMlrh3bu2v9GtZrMxOOmV7zLZd/wUCT06s2BytEyyJppEAt1ZaA371xqjULNWgIsbDJwJI+6izD+OS19dS6CvB1/de7FdvtpXVJvYvL+IlMwCUjKPse/4KQB6B3iTZAnw4b274unuvK3wxqgxmfnPVmNw0tHSCkZHBvLopChiwmRwkrCeBLozKtxrzMm+7ROoKoPuw9kaci3XrAvk6elx3DKyV4t87KGiclKyClibeYwfsws5XW3Cw82FEX38a1vhvQK8W+SzbU5rMFVBTYVxUdpUCR18jNWrWlBFtYmPNuXwVspeisurmRYXwkMTo+jdVv7ehENJoDuzilJjaoGf34YT+yl0DeDjmonccM9TBAaHNvvwVTVmtuQU1U50lX3MWNSje9eOxujMqCBG9PGnYwcrW+FmE9RUGuF5JkRrKn+/rabCErSVdQK38rdttftWnhvI57y/7vHPer+p6tz6lCv0GWOsVNV/GnTs0uy/w/Mprajm3XX7eG/9fqpMZq5J7M4D4yPo5mf/gVKi7ZBAbw/MJtjzHafXv0nH3A1UKQ86DL7OuKfdP8KKkPvt9RNlJ8nMPc7evEJyC4rBVIWXSzW9OrvRu4s73Tu50snNhDJZG6J1AtdcY5vzVa7G9MSuHcDNE9w6gKuHsc3Nw/L4PNvcPC3vq/t+y+MTOZD+pfGni7sx4CtmlnGnkWcn29R+loKyShatMQYnuZwZnJTUl85eMjhJnEsCvZ35aOly3FPf4RqPjbiabLgOqYt78wOz3vfXDeQz76/v+B1+e28zJjdrkNbGilTpX0L6Uig5ZHxmxATjmkXUZdDB9t0jh4rKee37LJZsP4yPhxt3WgYntZe5eoR1mh3oSqnJwN8AV+A9rfWL9exzDbAQ0MCvWuvrL3RMCfSWU1FtYtLr6+hCGf8ZlYtb9alzArO4SpGWX8H2I+VsO1JBcZg7sPMAABIBSURBVJXC7OpOZGgAg/t0Y1i/EHoFd0G5efwW0FZOSeBUzGY4nApp/zXC/eRRcOsIkZOMlnvERHC37f3zmUfL+OvKTFbtzifAx4N54/sxe2iPNjnQStheswJdKeUKZAETgFxgC3Cd1npXnX0igC+AcVrrE0qpIK31sQsdVwK9Za3NKuCW9zfz0IRI7h8fQY3JzLZDxazJOEZKZgG78koBCPHzrL0jZVS/AFlH80LMJji4CdK+NAZ9lR83LqJGTTHCve84m65StfXACf6yIoPN+4vo3tUYnDR9kAxOau+aG+gXAQu11pMszx8H0Fq/UGefl4AsrfV71hYlgd7y7v30F1btzmd8/yA27DlOaUUNri6KxJ5dGBsdRFJUIFHBvjLIpSlMNcaMmelfwu6v4fQJY7qG/tOMC6p9xoBr80ftaq1Zm1XAX1dmkn6klKhgXx6ZFMX4/kHy79ZONTfQrwIma63/YHl+EzBca31fnX2WYrTiR2F0yyzUWq+o51hzgbkAPXr0GHLgwIGmnZGwytGSCia9vg4PNxeSogIZGxXEqIgAOnm2zukB2ixTNexLMVruGd9AZSl07Ar9Lzda7r0uAZfm3YtvNmuSd+bx6vdZ7D9+iiE9u/DopCiGy+Ckdscegf4NUA1cA4QD64BYrXXx+Y4rLXT7qLDcJy6tOTuproC9PxjhnvktVJ8y1pcdMMOYsqH7iGZdi6g2mfl3ai5/+yGL/NJKkqICeWRSFANDZXBSe3GhQLemw/Qw0L3O83DLtrpygZ+11tXAfqVUFhCB0d8uHEhGadqZuydETzV+qsphz3dGt8y2T415eXxDYeAVRrdMeOI5k601eHhXF64f3oNZg8P4cKMxOGnqGxu4fFAoD02IbDuDukSLsKaF7obRnTIeI8i3ANdrrdPr7DMZ40LpLUqpAGAbEK+1LjzfcaWFLtqVypPGdA1pX0L298b9+X49jHCPmQUh8Y0OdzAWx35n3V7e35BDtcnMNUONwUnBDljFSdiHLW5bnAK8jtE//r7W+nml1HNAqtZ6mTK+z78CTAZMwPNa688udEwJdNFuVZRAxnKj5b53tTHYqmsf4x73gbMgeGCjw/1YWQVvrs7m//18EDdXxZyRvbl7TN9WO52yaDoZWCREa1VeZFxITfsS9q8DbYKASCPYY2Y1eq3Zg4XlvLYqi6WWwUl3jenLraN6yeAkJyKBLkRbcOq4cX972pdw4EdAQ3CM0XKPmWW04q2UcbSUl1dmsmr3MQJ9PZg3rh/XyuAkpyCBLkRbU5pnhHv6l3DoZ2NbSLwR7ANnQuceVh0mNaeIl1ZksjmniB5dvSyDk0JxkcFJbZYEuhBtWfEh2LXUaLkf+cXYFj7U6JYZeAV0uvCsmlprUrIKeGlFJrvzSonu5sujk6MYGyWDk9oiCXQhnEXRfkhfYrTcj+4EFPS4yGi5D5gBPkHnfavZrPl6xxFe/T6LA4XlJPbswoLLohnaq6v96hfNJoEuhDM6vscI97QvjaUJlQv0utgyl/t08K5/FGm1ycznWw7xxg97OFZWybjoIB6eGMWA0JaZHljYlgS6EM4uf5fRak/7Eor2GvPF9x1rhHv01HpXYTpdZeKDjTn8IyWbssoapg8KZf6ESHr6y+Ck1kwCXYj2Qms4usMI9vQvofigMfVx3/GWhTouAw/f372lpLyat9ft5f0f91Nj0swe1p154yIIksFJrZIEuhDtkdZw+BfLQh1LoPSwMbd9xASj5R45GTp41e5+rLSCN1bv4bPNh3BzVdw6qjd3jZbBSa2NBLoQ7Z3ZDLmbLXO5L4WT+eDuZYR6zCzoN8GYhwY4UHiKV7/PYtmvR/D1cOPupH7MGdnL+nVjRYuSQBdC/MZsggMbjZb7rq+gvBA6+Bp97TGzoM9YcOvAriOlvPxdJqszjhHk68G88RFcO7Q77q4yOMmRJNCFEPUz1UDOOmOJvd1fG/PMeHb+baGO3mPYcqiUv3ybQeqBE/T0NwYnXR4ng5McRQJdCNGwmirYt8ayUEcyVJWBlz/0n44eOJM1FRG89F02GUfL6B/SiUcnRZEUFSiDk+xMAl0I0TjVFZC9yuiWyVxhLNThE4zuP50NnmN4MtWLAycqGNarK49OjiJRBifZjQS6EKLpqsphz0qj5b7nO6ipQPuGsrvreP6SO5C1p7pzSUQgswaHcWn/YHxlicMWJYEuhLCNyjKjxZ7+pdGCN1VR4hnGqqoYcip9KXPpREhIGDERvUiI6odX5yBjfVV3uafdViTQhRC2d7rY6GtP/xJ9aDOqsvS8u2p3b5SXP3h1MfrlvfyNoPfyB6+ulp+ztrl3tOPJtB3NXVNUCCHO1bEzJNwACTegwLioevoE5lOFZOXk8GvmPvYePIhbxQkCzafo71lNH11JwOkSXIr2G4t7VJac//juXr+F++/C/8wvhDq/HM5sb+e/BCTQhRC24dYBfINx8Q0mutsAokeAyazZklNE8o48FqXlcbygCq8OrozvH8zU2BCS+nXGs6bUuBe+vMj483RRnednHhdC8QHjz4oGfgl07Pr7Vn9D3wbqjJZt66TLRQhhFzUmM5v3F/HNzjxWpB2l6FQVPh5uXNo/iKlxoYyODMDDzYrRqKYaOH3irPCv8wuhvOjc7RXF5z+eW8fzdPvUafmf/W3A3atJi3rbgvShCyFalRqTmU37CknekceK9KMUl1fj6+HGhIHBTIsL4eJ+gbZdLs9UY4T6OeF/5pdC0bnfEk6fOP/x3DzrhP9Z3UH1fRvw8rfZLwEJdCFEq1VtMvNj9nGSd+SxMv0opRU1dPJ0Y9LAbkyNC2FUvwDHTDdQ+0vg7PA/80uhnm8Jp4uB82Sqq8dvQT/6EWO1qSaQQBdCtAlVNWY2ZBfwzY48vk/Pp6yyhs5e7ky2hPtFffxxa81zyZhNRqifrzvozLeBxNsh4tImfYQEuhCizamsMbEu6zjJO47w/a58TlWZ6Ordgckx3ZgWG8LwPv64tsP5ZCTQhRBtWkW1ibVZRsv9h935lFeZCPDpwGUxIUyNC2For67tJtwl0IUQTuN0lYmUzGNGuGfkU1FtJsjXgymxRrgP6dHFqWeCbHagK6UmA38DXIH3tNYvnvX6HOCvwGHLpje11u9d6JgS6EKI5iqvqmF1xjG++TWPNZnHqKwxE9zJCPdpcaEkdO/sdOHerEBXSrkCWcAEIBfYAlyntd5VZ585QKLW+j5ri5JAF0LY0snKGn7YnU/yjjxSsgqoqjET6udZ23KP797ZKab6be7Q/2FAttZ6n+VgnwEzgF0XfJcQQtiRj4cbM+LDmBEfRllFNass4f7hphze27CfsM4dmRZnhHtsmJ9ThPvZrAn0MOBQnee5wPB69rtSKTUaozX/oNb60Nk7KKXmAnMBevTo0fhqhRDCCr6e7sxMCGdmQjglp6v5flc+yTuO8H8b9vP2un306OrF1LgQpsaGMDC0k9OEuzVdLlcBk7XWf7A8vwkYXrd7RSnlD5zUWlcqpe4ErtVaj7vQcaXLRQhhb8XlVXyXns83O/P4Mfs4JrOmd4A3Uy3dMtHdfFt9uDe3D/0iYKHWepLl+eMAWusXzrO/K1Cktfa70HEl0IUQjlR0qoqV6UdJ3pHHxr3HMWvoG+jN1LhQpsWFEBns6+gS69XcQHfD6EYZj3EXyxbgeq11ep19QrTWeZbHM4EFWusRFzquBLoQorU4frKSFWlGuP+8vxCzhshgH6bGhjI1LoR+QT6OLrGWLW5bnAK8jnHb4vta6+eVUs8BqVrrZUqpF4DpQA1QBNyttc640DEl0IUQrdGxsgpWpB3lmx15bMkpQmuI7uZruaAaSu8Ab4fWJwOLhBCiCfJLK1i+M4/kHXmkHjBmXxwQ0olpg4wLqj397R/uEuhCCNFMeSWnWb7zKN/sOMK2g8b86rFhfkyLC2FKbAjdu9pnoQwJdCGEsKHcE+V8awn3X3ONFZQGde/MtNgQpsSFENa55ZbCk0AXQogWcqionGRLt8zOw0a4D+7RmalxoUyNDaGbn6dNP08CXQgh7CDn+KnacN+VVwrA0F5dmBprdMsEdWp+uEugCyGEne0tOMnyHXkk78wj42gZSsGwXl2ZFhfC5JgQAn09mnRcCXQhhHCgPfllJO/M45sdeWQfO8mz0wdyy8heTTpWcyfnEkII0QwRwb78T7Av/3NpJFn5ZQQ1sXXeEAl0IYSwo5acUqAVr7YqhBCiMSTQhRDCSUigCyGEk5BAF0IIJyGBLoQQTkICXQghnIQEuhBCOAkJdCGEcBIS6EII4SQk0IUQwklIoAshhJOQQBdCCCchgS6EEE5CAl0IIZyEBLoQQjgJCXQhhHASEuhCCOEkrAp0pdRkpVSmUipbKfXYBfa7UimllVL1rncnhBCi5TQY6EopV2ARcBkwALhOKTWgnv18gQeAn21dpBBCiIZZ00IfBmRrrfdprauAz4AZ9ez3J+AvQIUN6xNCCGElawI9DDhU53muZVstpdRgoLvWOtmGtQkhhGiEZl8UVUq5AK8CD1mx71ylVKpSKrWgoKC5Hy2EEKIOawL9MNC9zvNwy7YzfIEYIEUplQOMAJbVd2FUa/2O1jpRa50YGBjY9KqFEEKcw5pA3wJEKKV6K6U6ALOBZWde1FqXaK0DtNa9tNa9gJ+A6Vrr1BapWAghRL0aDHStdQ1wH7AS2A18obVOV0o9p5Sa3tIFCiGEsI6bNTtprZcDy8/a9vR59k1qfllCCCEaS0aKCiGEk5BAF0IIJyGBLoQQTkICXQghnIQEuhBCOAkJdCGEcBJKa+2YD1aqADjgkA9vmgDguKOLsCE5n9ZNzqd1c+T59NRa1zvU3mGB3tYopVK11k4zz7ucT+sm59O6tdbzkS4XIYRwEhLoQgjhJCTQrfeOowuwMTmf1k3Op3VrlecjfehCCOEkpIUuhBBOQgJdCCGchAT6WZRSk5VSmUqpbKXUYxfY70qllK5vZabWxJrzUUpdo5TapZRKV0r9P3vX2BgNnY9SqodSao1SaptSaodSaooj6rSGUup9pdQxpVTaeV5XSqk3LOe6w7J2b6tlxfncYDmPnUqpjUqpQfausTEaOp86+w1VStUopa6yV23npbWWH8sP4ArsBfoAHYBfgQH17OcLrMNYnSnR0XU353yACGAb0MXyPMjRdTfzfN4B7rY8HgDkOLruC5zPaGAwkHae16cA3wIKY2nHnx1dczPPZ2Sd/84ua+vnY9nHFViNsV7EVY6uWVrovzcMyNZa79NaVwGfATPq2e9PwF+ACnsW1wTWnM8dwCKt9QkArfUxO9fYGNacjwY6WR77AUfsWF+jaK3XAUUX2GUG8JE2/AR0VkqF2Ke6xmvofLTWG8/8d4bRGAq3S2FNZMW/D8D9wH+BVvH/jQT674UBh+o8z7Vsq2X52ttda51sz8KaqMHzASKBSKXUj0qpn5RSk+1WXeNZcz4LgRuVUrkYrab77VNai7DmfNuq2zG+fbRZSqkwYCbwD0fXcoZVS9AJg1LKBXgVmOPgUmzJDaPbJQmjxbROKRWrtS52aFVNdx3wgdb6FaXURcDHSqkYrbXZ0YUJg1JqLEagX+zoWprpdWCB1tqslHJ0LYAE+tkOA93rPA+3bDvDF4gBUiz/gN2AZUqp6VrrVLtVab2GzgeMVt/PWutqYL9SKgsj4LfYp8RGseZ8bgcmA2itNymlPDEmUmoVX4kbyZrzbVOUUnHAe8BlWutCR9fTTInAZ5YsCACmKKVqtNZLHVWQdLn83hYgQinVWynVAZgNLDvzota6RGsdoLXupbXuhdEP2FrDHBo4H4ulGK1zlFIBGF0w++xZZCNYcz4HgfEASqn+gCdQYNcqbWcZcLPlbpcRQInWOs/RRTWVUqoH8CVwk9Y6y9H1NJfWunedLPgPcI8jwxykhf47WusapdR9wEqMq9fva63TlVLPAala67PDo1Wz8nxWAhOVUrsAE/BIa205WXk+DwHvKqUexLhAOkdbbkdobZRS/8L4ZRpg6fN/BnAH0FovxrgGMAXIBsqBWx1TqXWsOJ+nAX/gLUurtka3whkLz7DifFodGfovhBBOQrpchBDCSUigCyGEk5BAF0IIJyGBLoQQTkICXQghnIQEuhBCOAkJdCGEcBL/H68rEPNo83ShAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "plot(training_logs.logs, [\"eval_accuracy\"], [\"Evaluation accuracy\"])"
      ],
      "metadata": {
        "id": "7cfERwdudGXy",
        "outputId": "f7dcc372-5f90-4447-e814-39fcf0e94fbd",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 265
        }
      },
      "execution_count": 89,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 432x288 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAXoAAAD4CAYAAADiry33AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAdRklEQVR4nO3dfXhU5bnv8e+dZMgLhJAAUksCQYUC8ipTULGCUlrqoVpUFLVV3Fq2ttqjtj3V1qOIL9tt7QZqrRa9bMWrllpbWrRatxGono1sCUq1wBYiogQRYxICCoG83OePmYyTkJAJCZmw+H2ua65ZL8+a3A8Jv/XMs1Yy5u6IiEhwpSS7ABERObIU9CIiAaegFxEJOAW9iEjAKehFRAIuLdkFNNWnTx8vLCxMdhkiIkeVtWvXfuzufZvb1+WCvrCwkOLi4mSXISJyVDGz91rap6kbEZGAU9CLiAScgl5EJOC63By9yLGupqaG0tJSqqurk12KdEEZGRnk5+cTCoUSPkZBL9LFlJaWkp2dTWFhIWaW7HKkC3F3ysvLKS0tZdCgQQkfp6kbkS6murqa3r17K+TlIGZG79692/xuT0Ev0gUp5KUlh/OzoaAXEQk4Bb2IHCQ1NZUxY8bEHvfee+9hvc7kyZMP+xcgV65cyapVq2LrDz/8MIsXLz6s1zrW6WKsiBwkMzOTdevWJbWGlStX0qNHD04//XQArrnmmqTWczhqa2tJS0t+zGpELyIJ+dvf/sbMmTNj6ytXrmT69OkAXHvttYTDYU4++WRuv/32Zo/v0aNHbPnpp59m9uzZADzzzDNMmDCBsWPH8uUvf5mdO3eydetWHn74YebPn8+YMWN45ZVXmDt3Lvfffz8A69at49RTT2XUqFHMmDGDyspKIPIO4kc/+hHjx49nyJAhvPLKKwfV8cknnzBlyhROOeUURo4cyV/+8pfYvsWLFzNq1ChGjx7Nt771LQB27tzJjBkzGD16NKNHj2bVqlVs3bqVESNGxI67//77mTt3bqyGG264gXA4zMKFC5vtX0MdV155JSNHjmTUqFH88Y9/5LHHHuOGG26Ive4jjzzCjTfemNg36BCSf6oRkRbd8cx6Nnywu0Nfc/jne3L7108+ZJt9+/YxZsyY2Pott9zCBRdcwJw5c/j000/p3r07v//975k1axYAd999N3l5edTV1TFlyhTefPNNRo0alVA9Z5xxBqtXr8bMePTRR7nvvvv42c9+xjXXXEOPHj34wQ9+AMBLL70UO+byyy/ngQceYNKkSdx2223ccccdLFiwAIiMol977TWee+457rjjDoqKihp9vYyMDJYuXUrPnj35+OOPOfXUUzn33HPZsGEDd911F6tWraJPnz5UVFQA8L3vfY9JkyaxdOlS6urq+OSTT2InlpYcOHAgNmVVWVnZbP/uvPNOcnJyeOutt2LtQqEQd999Nz/96U8JhUL8+te/5le/+lVC/46HoqAXkYO0NHUzbdo0nnnmGS688EL++te/ct999wHw1FNPsWjRImpra9mxYwcbNmxIOOhLS0u5+OKL2bFjBwcOHGj1/vCqqip27drFpEmTALjiiisavdM4//zzARg3bhxbt2496Hh358c//jEvv/wyKSkpbN++nZ07d7J8+XJmzpxJnz59AMjLywNg+fLlsWsDqamp5OTktBr0F198cav9KyoqYsmSJbF2ubm5AJx99tk8++yzDBs2jJqaGkaOHHnIr5UIBb1IF9bayLuzzZo1i1/84hfk5eURDofJzs7m3Xff5f7772fNmjXk5uYye/bsZu/zjr8tMH7/9ddfz0033cS5557LypUrY1Mghys9PR2IhHJtbe1B+3/7299SVlbG2rVrCYVCFBYWtvm+9LS0NOrr62PrTY/v3r17bLmt/bv66qu55557GDp0KFdeeWWb6mqJ5uhFJGGTJk3i9ddf55FHHolN2+zevZvu3buTk5PDzp07ef7555s9tl+/fmzcuJH6+nqWLl0a215VVUX//v0BePzxx2Pbs7Oz2bNnz0Gvk5OTQ25ubmz+/YknnoiN7hNRVVXFcccdRygUYsWKFbz3XuSv+5599tn84Q9/oLy8HCA2dTNlyhQeeughAOrq6qiqqqJfv3589NFHlJeXs3//fp599tlDfr3m+jd16lQefPDB2HrDu4QJEyawbds2nnzySS655JKE+3UoCnoROUjDHH3D4+abbwYio+Tp06fz/PPPxy7Ejh49mrFjxzJ06FAuvfRSJk6c2Oxr3nvvvUyfPp3TTz+d448/PrZ97ty5zJw5k3HjxsWmTQC+/vWvs3Tp0tjF2HiPP/44P/zhDxk1ahTr1q3jtttuS7hvl112GcXFxYwcOZLFixczdOhQAE4++WR+8pOfMGnSJEaPHs1NN90EwMKFC1mxYgUjR45k3LhxbNiwgVAoxG233cb48eOZOnVq7DWa01L/br31ViorKxkxYgSjR49mxYoVsX0XXXQREydOjE3ntJe5e+uNzKYBC4FU4FF3v7fJ/oHAY0BfoAL4pruXRvddAdwabXqXuz/OIYTDYdcHj8ixbOPGjQwbNizZZUgSTZ8+nRtvvJEpU6Y0u7+5nxEzW+vu4ebatzqiN7NU4EHga8Bw4BIzG96k2f3AYncfBcwD/i16bB5wOzABGA/cbmYdc4oSEQmYXbt2MWTIEDIzM1sM+cORyMXY8UCJu28BMLMlwHnAhrg2w4GbossrgD9Hl78KvOjuFdFjXwSmAb9rf+kiIsHSq1cvNm3a1OGvm8gcfX9gW9x6aXRbvH8A50eXZwDZZtY7wWMxszlmVmxmxWVlZYnWLhJYiUypyrHpcH42Oupi7A+ASWb2BjAJ2A7UJXqwuy9y97C7h/v2bfZDzEWOGRkZGZSXlyvs5SANf48+IyOjTcclMnWzHSiIW8+Pbov/4h8QHdGbWQ/gAnffZWbbgclNjl3ZpgpFjjH5+fmUlpaid7fSnIZPmGqLRIJ+DTDYzAYRCfhZwKXxDcysD1Dh7vXALUTuwAF4Abgn7gLsV6L7RaQFoVCoTZ8eJNKaVqdu3L0WuI5IaG8EnnL39WY2z8zOjTabDLxtZpuAfsDd0WMrgDuJnCzWAPMaLsyKiEjnSOg++s6k++hFRNquXffRi4jI0U1BLyIScAp6EZGAU9CLiAScgl5EJOAU9CIiAaegFxEJOAW9iEjAKehFRAJOQS8iEnAKehGRgFPQi4gEnIJeRCTgFPQiIgGnoBcRCTgFvYhIwCnoRUQCTkEvIhJwCnoRkYBT0IuIBJyCXkQk4BT0IiIBp6AXEQk4Bb2ISMAp6EVEAk5BLyIScAp6EZGAU9CLiAScgl5EJOAU9CIiAaegFxEJOAW9iEjAKehFRAIuoaA3s2lm9raZlZjZzc3sH2BmK8zsDTN708zOiW4vNLN9ZrYu+ni4ozsgIiKHltZaAzNLBR4EpgKlwBozW+buG+Ka3Qo85e4Pmdlw4DmgMLrvHXcf07Fli4hIohIZ0Y8HStx9i7sfAJYA5zVp40DP6HIO8EHHlSgiIu2RSND3B7bFrZdGt8WbC3zTzEqJjOavj9s3KDql83cz+1J7ihURkbbrqIuxlwC/cfd84BzgCTNLAXYAA9x9LHAT8KSZ9Wx6sJnNMbNiMysuKyvroJJERAQSC/rtQEHcen50W7yrgKcA3P1VIAPo4+773b08un0t8A4wpOkXcPdF7h5293Dfvn3b3gsREWlRIkG/BhhsZoPMrBswC1jWpM37wBQAMxtGJOjLzKxv9GIuZnYCMBjY0lHFi4hI61q968bda83sOuAFIBV4zN3Xm9k8oNjdlwHfBx4xsxuJXJid7e5uZmcC88ysBqgHrnH3iiPWGxEROYi5e7JraCQcDntxcXGyyxAROaqY2Vp3Dze3T78ZKyIScAp6EZGAU9CLiAScgl5EJOAU9CIiAaegFxEJOAW9iEjAKehFRAJOQS8iEnAKehGRgFPQi4gEnIJeRCTgFPQiIgGnoBcRCbhW/x69iIh0HHdnf209+2vq2V9bR3X0eX9tPaHUFL7wuewO/5oKehE55rg7B+rqY4FbXRMJ2qbBu79hezOhXB23r7q27tBt4pYP1Na3WNfYAb1Y+p2JHd5fBb2IJE1NLGzrqI4L1uYCsmmb/XFtGgVso+BtOcTb85lLKQYZoVTS01JIT0slIxR5Tg+lkJGWSvf0NPK6p5DeXJu0lM+OjbZPj+7r3aNbx/3jxlHQi0iHc3de2vgRT6x+j117D7Q4yq2rb98n3EVCNBqcTUIzI5RCz8xQ42BtFLKNn5tt0xDQTdqkpRhm1kH/Wkeegl5EOoy7s+Ltj1hQtJk3S6vo3yuTk47rkdBotqU2TUfMDW3S01KOqrBNJgW9iLSbu7Py7TIWFG3iH6VVFORlct8Fo5hxSn9Cqbq5L9kU9CJy2Nydv28qY0HRZtZt20V+bib/fsFIzj8lXwHfhSjoRaTN3J2XN3/MgqJNvPH+Lvr3yuTfzh/JBafk0y1NAd/VKOhFJGHuzv8r+Zj5L27i9WjA3zNjJBeOU8B3ZQp6EWmVu7PqnXLmv7iJ4vcqOT4ng7u+MYKZ4XzS01KTXZ60QkEvIi1yd159p5wFRZt5bWsFx+dkcOc3RnCRAv6ooqAXkWa9+k4584s28dq7FfTrmc68807m4i8WKOCPQgp6EWlk9ZZyFhRtYvWWCo7LTmfu14cza/wAMkIK+KOVgl5EAHjt3Qrmv7iJV7eU0zc7ndumD+fSCQr4IFDQixzjirdWML9oE/9VUk6fHun83+nDuUwBHygKepFj1Nr3KlhQtJlXNn9Mnx7duPV/DeOyCQPJ7KaADxoFvcgx5vX3K5n/4iZe2fwxvbt34yfnDOObpyrgg0xBL3KMeOP9ShYUbebvm8rI696NW742lG+dNpCsboqBoNN3WCTg/rFtF/OLNrHy7TJys0L8aNpQLj9tIN3T9d//WJHQd9rMpgELgVTgUXe/t8n+AcDjQK9om5vd/bnovluAq4A64Hvu/kLHlS8iLXmzdBcLijaz/H8+oldWiP8z7QtccVqhAv4Y1Op33MxSgQeBqUApsMbMlrn7hrhmtwJPuftDZjYceA4ojC7PAk4GPg8UmdkQd6/r6I6ISMRbpVUsfGkTRRsjAf/Dr36BK04vpIcC/piVyHd+PFDi7lsAzGwJcB4QH/QO9Iwu5wAfRJfPA5a4+37gXTMrib7eqx1Qu4jE+ef2KhYUbaZo405yMkP84CtDuOL0QrIzQskuTZIskaDvD2yLWy8FJjRpMxf4TzO7HugOfDnu2NVNju1/WJWKSLPWf1DFwqLN/OeGnfTMSOOmqUOYPbGQngp4ieqo93KXAL9x95+Z2WnAE2Y2ItGDzWwOMAdgwIABHVSSSLBt3LGbBUWbeGH9TrIz0rjhy4O5cuIgcjIV8NJYIkG/HSiIW8+Pbot3FTANwN1fNbMMoE+Cx+Lui4BFAOFwuH2fFiwScP/z4W4WFm3m+X9+SHZ6Gv97ymD+5QwFvLQskaBfAww2s0FEQnoWcGmTNu8DU4DfmNkwIAMoA5YBT5rZfxC5GDsYeK2Dahc5prz94R5+/tJm/vrWDnqkp/G9s0/iqjNOICdLAS+H1mrQu3utmV0HvEDk1snH3H29mc0Dit19GfB94BEzu5HIhdnZ7u7AejN7isiF21rgu7rjRqRtNu/cw4KXNvPcWzvICqVy3VkncfWXBtErq1uyS5OjhEXyuOsIh8NeXFyc7DJEkq7koz0sfKmEZ9/8gKxQKrMnFnL1GSeQ210BLwczs7XuHm5un26sFeliSj76hJ+/tJln3vyAzFAq10w6kW9/6QTyFPBymBT0Il3ElrJIwC/7xwdkhFL51zNPZM6ZCnhpPwW9SJK9+/GnPPDSZv68bjvpaal8+0snMOfME+jdIz3ZpUlAKOhFkmTrx5/y8+Wb+fMb2+mWlsLV0YDvo4CXDqagF+lk75V/ygPLS1j6xnbSUox/mTiIf510In2zFfByZCjoRTrJ++V7eWD5Zv4UDfgrTivkmskncFx2RrJLk4BT0IscYdsq9vKL5SX88fVSUlKMy08byLWTTuS4ngp46RwKepEjZFvFXn65soQ/FEcC/punDuTaySfSTwEvnUxBL9LBtu/axy+Wl/D02m0YxmUTBnDt5JP4XI4CXpJDQS/SQbbv2scvV5TwVHEk4Gd9cQDfOetEjs/JTHZpcoxT0Iu0046qfTy4ooTfr4l8bMNF4QK+e9ZJfL6XAl66BgW9yGH6sKqaX64sYclr23CcmeECvjP5RPJzs5JdmkgjCvoAc3fKPtnPtop9bKvYG3lU7uX9ir1sq9jHnuoa0kOppKelkJ6WQkZsOZWMUOQ5PZRCRvT5UG3SE2iTkZZKKNUws2T/07TLzt3VPLTyHZ587X3q652Z4Xy+e9ZJCnjpshT0R7k91TWRIK/cGxfm+2KhXl1T36j9cdnpFORlMX5QHjmZIfbX1rO/to79NdHn2nqqa+oo/7SW/TX1VMftq46ut+cPnprR8skgfnujE0zcc6Nj49unkhF9bqlNWmpKu/6tP9pdzS/jAv7CcZGAL8hTwEvXpqDv4g7U1vPBrn2RUXjl3s9G59Fgr9xb06h9dnoaBXlZnNC3O5O/0JeCvCwKcrMoyMsiPzeTjFBqu+pxd2rrneqauuhJInJiiD8ZxJ8wGtrsb1iuqaM6br3p6+yrqWPXvgOfvU5N4zbtkZpiLZ4M0lt5N7Nrbw1/er2U2nrnglP6c91ZgxnQWwEvRwcFfZLV1zdMr0SnVcobj84/3F1NfdwIultqCv1zMynIy2Jk/xwK8rIYEAvzTHIyQ0d0asTMCKUaodQUso/YV2meu3924qht5uRSU9/k5NF4X3UrbT7ZX0v5JwcatWs4ITnOeWP6c/3ZJzGwd/dO7rlI+yjoO8Hu6hreL99LacOIPDZPvpfSyn0HjVQ/1zODgrxMTj2xd2w0XpCbyYDeWfTLziAl5eie4z5cZkZGKDX6rqRzPz7P3Y/6awty7FLQd4D9tXVsr9zHtsrIFEtpk4ueVfsaT6/0zEhjQO8shvTLZsqwfrEgL8jLon+v9k+vSMdTyMvRTEGfgPp6Z+ee6kbz45FAj4zOP9xd3egCZbe0FPJzMxmQl8XYglwK8iLL+dHReU6mPsxZRDqPgj6qam9NoymVyPI+Siv2UrprHwfiplfM4PieGeTnZXH6iX0ic+R5mbH58r490o/Z6RUR6XqOmaCvrqmjtDIyAi+t+GxapSHc91TXNmrfKytEQW4Ww47vydST+1GQG73omZfF53tlkJ6m6RUROToEJujr650Pd1fHjcj3fRbolXvZuXt/o/bpaSmxufHwwNzo7YdZsdF5doamV0QkGAIT9Dv3VHP6vctj6ykGx+dkUpCXyZmD+352G2JeJgW5WfTNTtcFNhE5JgQm6PtlZ3DPjJGxMD8+J5Nuae37TUgRkSAITNCnpBiXThiQ7DJERLocDXlFRAJOQS8iEnAKehGRgFPQi4gEnIJeRCTgFPQiIgGnoBcRCTgFvYhIwCnoRUQCLqGgN7NpZva2mZWY2c3N7J9vZuuij01mtituX13cvmUdWbyIiLSu1T+BYGapwIPAVKAUWGNmy9x9Q0Mbd78xrv31wNi4l9jn7mM6rmQREWmLREb044ESd9/i7geAJcB5h2h/CfC7jihORETaL5Gg7w9si1svjW47iJkNBAYBy+M2Z5hZsZmtNrNvtHDcnGib4rKysgRLFxGRRHT0xdhZwNPuXhe3baC7h4FLgQVmdmLTg9x9kbuH3T3ct2/fDi5JROTYlkjQbwcK4tbzo9uaM4sm0zbuvj36vAVYSeP5exEROcISCfo1wGAzG2Rm3YiE+UF3z5jZUCAXeDVuW66ZpUeX+wATgQ1NjxURkSOn1btu3L3WzK4DXgBSgcfcfb2ZzQOK3b0h9GcBS9zd4w4fBvzKzOqJnFTujb9bR0REjjxrnMvJFw6Hvbi4ONlliIgcVcxsbfR66EH0m7EiIgGnoBcRCTgFvYhIwCnoRUQCTkEvIhJwCnoRkYBT0IuIBJyCXkQk4BT0IiIBp6AXEQk4Bb2ISMAp6EVEAk5BLyIScAp6EZGAU9CLiAScgl5EJOAU9CIiAaegFxEJOAW9iEjAKehFRAJOQS8iEnAKehGRgFPQi4gEnIJeRCTgFPQiIgGnoBcRCTgFvYhIwCnoRUQCTkEvIhJwCnoRkYBT0IuIBJyCXkQk4BT0IiIBl1DQm9k0M3vbzErM7OZm9s83s3XRxyYz2xW37woz2xx9XNGRxYuISOvSWmtgZqnAg8BUoBRYY2bL3H1DQxt3vzGu/fXA2OhyHnA7EAYcWBs9trJDeyEiIi1KZEQ/Hihx9y3ufgBYApx3iPaXAL+LLn8VeNHdK6Lh/iIwrT0Fi4hI2yQS9P2BbXHrpdFtBzGzgcAgYHlbjjWzOWZWbGbFZWVlidQtIiIJ6uiLsbOAp929ri0Hufsidw+7e7hv374dXJKIyLEtkaDfDhTEredHtzVnFp9N27T1WBEROQISCfo1wGAzG2Rm3YiE+bKmjcxsKJALvBq3+QXgK2aWa2a5wFei20REpJO0eteNu9ea2XVEAjoVeMzd15vZPKDY3RtCfxawxN097tgKM7uTyMkCYJ67V3RsF0RE5FAsLpe7hHA47MXFxckuQ0TkqGJma9093Nw+/WasiEjAKehFRAJOQS8iEnAKehGRgFPQi4gEnIJeRCTgFPQiIgGnoBcRCTgFvYhIwCnoRUQCTkEvIhJwCnoRkYBT0IuIBJyCXkQk4BT0IiIBp6AXEQk4Bb2ISMAp6EVEAk5BLyIScAp6EZGAU9CLiAScgl5EJODM3ZNdQyNmVga8l+w62qAP8HGyi+hA6k/Xpv50bcnsz0B379vcji4X9EcbMyt293Cy6+go6k/Xpv50bV21P5q6EREJOAW9iEjAKejbb1GyC+hg6k/Xpv50bV2yP5qjFxEJOI3oRUQCTkEvIhJwCvoEmdk0M3vbzErM7OZDtLvAzNzMutwtVvES6Y+ZXWRmG8xsvZk92dk1tkVr/TGzAWa2wszeMLM3zeycZNSZKDN7zMw+MrN/trDfzOzn0f6+aWandHaNbZFAfy6L9uMtM1tlZqM7u8a2aK0/ce2+aGa1ZnZhZ9XWLHfXo5UHkAq8A5wAdAP+AQxvpl028DKwGggnu+729AcYDLwB5EbXj0t23e3szyLg2ujycGBrsutupU9nAqcA/2xh/znA84ABpwL/neya29mf0+N+1r52tPcn2iYVWA48B1yYzHo1ok/MeKDE3be4+wFgCXBeM+3uBP4dqO7M4g5DIv35NvCgu1cCuPtHnVxjWyTSHwd6RpdzgA86sb42c/eXgYpDNDkPWOwRq4FeZnZ851TXdq31x91XNfysERko5XdKYYcpge8PwPXAH4Gk/99R0CemP7Atbr00ui0m+ta5wN3/2pmFHaZW+wMMAYaY2X+Z2Wozm9Zp1bVdIv2ZC3zTzEqJjLCu75zSjphE+ny0uorIu5Wjlpn1B2YADyW7FoC0ZBcQBGaWAvwHMDvJpXSkNCLTN5OJjK5eNrOR7r4rqVUdvkuA37j7z8zsNOAJMxvh7vXJLkw+Y2ZnEQn6M5JdSzstAH7k7vVmluxaFPQJ2g4UxK3nR7c1yAZGACuj39TPAcvM7Fx3L+60KhPXWn8gMkL8b3evAd41s01Egn9N55TYJon05ypgGoC7v2pmGUT+AFXS31YfpkT6fFQxs1HAo8DX3L082fW0UxhYEs2DPsA5Zlbr7n9ORjGauknMGmCwmQ0ys27ALGBZw053r3L3Pu5e6O6FROYYu2rIQyv9ifozkdE8ZtaHyFTOls4ssg0S6c/7wBQAMxsGZABlnVplx1oGXB69++ZUoMrddyS7qMNlZgOAPwHfcvdNya6nvdx9UFwePA18J1khDxrRJ8Tda83sOuAFIlfSH3P39WY2Dyh296ah0qUl2J8XgK+Y2QagDvhhVx1lJdif7wOPmNmNRC7MzvborRFdkZn9jsiJtk/0usLtQAjA3R8mcp3hHKAE2AtcmZxKE5NAf24DegO/jI6Ca70L/hXIBgn0p0vRn0AQEQk4Td2IiAScgl5EJOAU9CIiAaegFxEJOAW9iEjAKehFRAJOQS8iEnD/H1giwXAGl6bvAAAAAElFTkSuQmCC\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Predict for new data\n",
        "\n",
        "Create `Dataset` with new examples. We'll store these as \"text\" as in the existing dataset so that our tokenizer will continue to work on this new dataset."
      ],
      "metadata": {
        "id": "8IVZNY1W_R-p"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "examples = datasets.Dataset.from_dict({\n",
        "    \"text\": [\n",
        "        \"<TARGET>Tuote A</TARGET> on loistava\",\n",
        "        \"<TARGET>Tuote B</TARGET> on surkea\",\n",
        "        \"<TARGET>Tuote C</TARGET> on sininen\",\n",
        "    ]\n",
        "})"
      ],
      "metadata": {
        "id": "y6FyJ_Yg3F7k"
      },
      "execution_count": 143,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Invoke the `Dataset.map()` function with our tokenizer, as above."
      ],
      "metadata": {
        "id": "7XvkghUs_iqN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "examples = examples.map(tokenize)"
      ],
      "metadata": {
        "id": "ACz4uO5D5_aG",
        "outputId": "7e601bc8-3fd6-4780-c296-e75ffb2df8c6",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 49,
          "referenced_widgets": [
            "747c02f9d9254b09ba811633440fba37",
            "6ec322204cef4463b88c75fd850d0d50",
            "bd84a09a2be84f6fb732c73f699d70f8",
            "e6545999e0634369992fb30021ab5e9c",
            "f46c70c3f68947bb8f3249850ca2c3fe",
            "e02689de93844bb2b13934cd6e7106f6",
            "448bf4d5f9894dd6927b72a5d4e859de",
            "e65a4ba44fec4506bc6001bdfc9930a0",
            "f581d78126b34ea7b673bbf835175e35",
            "a6edf96ffb24434eae98dfb74f2c83e1",
            "4c41ba8e69da487e820fbe58b85f6849"
          ]
        }
      },
      "execution_count": 146,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "  0%|          | 0/3 [00:00<?, ?ex/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "747c02f9d9254b09ba811633440fba37"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's confirm that what we got matches up with the structure of the dataset we created previously:"
      ],
      "metadata": {
        "id": "bjKXbgGT_oF3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "pprint(examples)\n",
        "\n",
        "pprint(dataset[\"test\"])"
      ],
      "metadata": {
        "id": "5EVegJpI3L1l",
        "outputId": "f5c54127-ba72-45b3-c950-de75815bba82",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 163,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Dataset({\n",
            "    features: ['text', 'input_ids', 'token_type_ids', 'attention_mask'],\n",
            "    num_rows: 3\n",
            "})\n",
            "Dataset({\n",
            "    features: ['label', 'text', 'input_ids', 'token_type_ids', 'attention_mask'],\n",
            "    num_rows: 343\n",
            "})\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "(`labels` are missing, but that's OK -- we won't be training on this data, so labels are not expected.)"
      ],
      "metadata": {
        "id": "7dAdwKf7_zmG"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Predict for the new examples and grab the outputs"
      ],
      "metadata": {
        "id": "0wd5ZS-z_7MC"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "outputs = trainer.predict(examples)\n",
        "outputs"
      ],
      "metadata": {
        "id": "nhOQNgr7645D",
        "outputId": "c6fb7851-d967-49d0-bb0b-3fc5a0b3dc7e",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 131
        }
      },
      "execution_count": 164,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='27' max='11' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [11/11 39:33]\n",
              "    </div>\n",
              "    "
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "PredictionOutput(predictions=array([[ 3.1074188 , -0.771859  , -1.4234914 , -0.68315864],\n",
              "       [-0.48153913,  3.132214  , -1.4463739 , -0.4061813 ],\n",
              "       [ 1.1090422 , -0.59848887, -1.832591  ,  1.3392398 ]],\n",
              "      dtype=float32), label_ids=None, metrics={'test_runtime': 0.0388, 'test_samples_per_second': 77.393, 'test_steps_per_second': 25.798})"
            ]
          },
          "metadata": {},
          "execution_count": 164
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can get the actual predictions (logits) as `outputs.predictions`:"
      ],
      "metadata": {
        "id": "sRQehK0nAGeN"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "outputs.predictions"
      ],
      "metadata": {
        "id": "M8eRkyVXAMqt",
        "outputId": "d70a2902-e2df-45c7-feb9-0504f64120af",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 165,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[ 3.1074188 , -0.771859  , -1.4234914 , -0.68315864],\n",
              "       [-0.48153913,  3.132214  , -1.4463739 , -0.4061813 ],\n",
              "       [ 1.1090422 , -0.59848887, -1.832591  ,  1.3392398 ]],\n",
              "      dtype=float32)"
            ]
          },
          "metadata": {},
          "execution_count": 165
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We can then `argmax` these along the correct axis to get the index of the label with the largest predicted output value for each example as follows:"
      ],
      "metadata": {
        "id": "h4LZz_3BAO-r"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "predicted_indices = outputs.predictions.argmax(axis=1)\n",
        "predicted_indices"
      ],
      "metadata": {
        "id": "eiTnm65j9dZ5",
        "outputId": "e8ce36a9-7252-4c96-deb0-f8bfb128a233",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 166,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([0, 1, 3])"
            ]
          },
          "metadata": {},
          "execution_count": 166
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's print the name of the predicted label along with the input text to see if the predictions make sense:"
      ],
      "metadata": {
        "id": "bixY-1N5Abk3"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "for text, index in zip(examples[\"text\"], predicted_indices):\n",
        "  print(text, \"->\", label_names[index])"
      ],
      "metadata": {
        "id": "j9X0rJB_9g0l",
        "outputId": "4f2f6e2a-2547-4ccc-947a-004fb63ed938",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 168,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "<TARGET>Tuote A</TARGET> on loistava -> positive\n",
            "<TARGET>Tuote B</TARGET> on surkea -> negative\n",
            "<TARGET>Tuote C</TARGET> on sininen -> neither\n"
          ]
        }
      ]
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [
        "wo13ZXoZYB6J",
        "AiX7WeW2X5gv",
        "Tmj91CdCT-fJ"
      ],
      "name": "hf_trainer_bert_sentiment_target.ipynb",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "50d593ed27aa4975bbbe1be40f847361": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e6c35ef27050487d893ea52b0d246fac",
              "IPY_MODEL_7497f66b64a64ef89ed4a0c4900a1bf0",
              "IPY_MODEL_e7bdc50578764db0a39eee44cae7fffa"
            ],
            "layout": "IPY_MODEL_13d8146a1a0643b9803172d718c13599"
          }
        },
        "e6c35ef27050487d893ea52b0d246fac": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_30f6173dac894bcfa8365b15e63b0083",
            "placeholder": "​",
            "style": "IPY_MODEL_e30d5391d3664fb39728e1eb389289f8",
            "value": "Downloading data files: 100%"
          }
        },
        "7497f66b64a64ef89ed4a0c4900a1bf0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f131fdfefcd646b0aa617d987dc90b75",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_fa1b41850b5b422f9de598e687dba64b",
            "value": 1
          }
        },
        "e7bdc50578764db0a39eee44cae7fffa": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2057f4bf3a90494088dab00e16f4c3c5",
            "placeholder": "​",
            "style": "IPY_MODEL_cdd380742b1043ad9ba02c09ef29c197",
            "value": " 1/1 [00:00&lt;00:00,  6.65it/s]"
          }
        },
        "13d8146a1a0643b9803172d718c13599": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "30f6173dac894bcfa8365b15e63b0083": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e30d5391d3664fb39728e1eb389289f8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f131fdfefcd646b0aa617d987dc90b75": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fa1b41850b5b422f9de598e687dba64b": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2057f4bf3a90494088dab00e16f4c3c5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cdd380742b1043ad9ba02c09ef29c197": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "434964fddb0644f8800559aa4a4313cd": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_3f45f6f7c34c4887bcdca515df276271",
              "IPY_MODEL_069a38cd37e04b64be5d1ad4b59f26b5",
              "IPY_MODEL_99eaa73e2fda48a4991db78640469ba4"
            ],
            "layout": "IPY_MODEL_31ef362c75ee4c5889ef2cbb70382531"
          }
        },
        "3f45f6f7c34c4887bcdca515df276271": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e6d51fdbfcc247938a5b1aae4ed1162f",
            "placeholder": "​",
            "style": "IPY_MODEL_2fb9fe33de8d4d16897d25abcda2e562",
            "value": "Extracting data files: 100%"
          }
        },
        "069a38cd37e04b64be5d1ad4b59f26b5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d2c1b9dd53f1427b9723fef8a5cb1c3f",
            "max": 1,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_133fdb0e0125418eb0be15c912a5e40d",
            "value": 1
          }
        },
        "99eaa73e2fda48a4991db78640469ba4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7eaa16fb81ba4896a1592538af246d21",
            "placeholder": "​",
            "style": "IPY_MODEL_b18c388639cd4010a0e8020b8bf6a67c",
            "value": " 1/1 [00:00&lt;00:00,  7.38it/s]"
          }
        },
        "31ef362c75ee4c5889ef2cbb70382531": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e6d51fdbfcc247938a5b1aae4ed1162f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2fb9fe33de8d4d16897d25abcda2e562": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d2c1b9dd53f1427b9723fef8a5cb1c3f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "133fdb0e0125418eb0be15c912a5e40d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "7eaa16fb81ba4896a1592538af246d21": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b18c388639cd4010a0e8020b8bf6a67c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f3dde39fddaf450a98df72c3cc9bc517": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_3634fb01fbfa4145b2e848e643354cc9",
              "IPY_MODEL_a8049da7abf743d9b8cb78785ffabdf1",
              "IPY_MODEL_e03ee90340954b0bacc52e763f75f822"
            ],
            "layout": "IPY_MODEL_5e755e85ff1040c9bd0b0ae2fe226894"
          }
        },
        "3634fb01fbfa4145b2e848e643354cc9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_dc8516a00cb9439eb28123ec24029524",
            "placeholder": "​",
            "style": "IPY_MODEL_afbaa639e9ce4075aa785270ef4fcd99",
            "value": "100%"
          }
        },
        "a8049da7abf743d9b8cb78785ffabdf1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_10e47ff402154ba9801330de0d72f537",
            "max": 3,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_fe876839751545a6a593832de77e5299",
            "value": 3
          }
        },
        "e03ee90340954b0bacc52e763f75f822": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_f809c868c6a449f5ba36ff5808277659",
            "placeholder": "​",
            "style": "IPY_MODEL_df4c2b0ea4e2435ebc47f98951bf20d8",
            "value": " 3/3 [00:00&lt;00:00, 33.51it/s]"
          }
        },
        "5e755e85ff1040c9bd0b0ae2fe226894": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dc8516a00cb9439eb28123ec24029524": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "afbaa639e9ce4075aa785270ef4fcd99": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "10e47ff402154ba9801330de0d72f537": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fe876839751545a6a593832de77e5299": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "f809c868c6a449f5ba36ff5808277659": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "df4c2b0ea4e2435ebc47f98951bf20d8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1ad9b0850ab94a938b61de6bed990482": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_da09dde448ee49b18c34eb2fb6e2a2f2",
              "IPY_MODEL_40d1a7eeac9c4fbbb14ea151c01a2357",
              "IPY_MODEL_4fe3f1b32997464080d6456a66dd2cb5"
            ],
            "layout": "IPY_MODEL_291af4a7e0254ccea17b9fa0827ff300"
          }
        },
        "da09dde448ee49b18c34eb2fb6e2a2f2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b359af007c8849d4a8a0187bedd4ab2c",
            "placeholder": "​",
            "style": "IPY_MODEL_16e48341d7724f4f927b978c745108a5",
            "value": "100%"
          }
        },
        "40d1a7eeac9c4fbbb14ea151c01a2357": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6da1d8566ac441cdadbb8b211b9b164a",
            "max": 2743,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_cc70cfe677124a4d9df99c0fc327ff44",
            "value": 2743
          }
        },
        "4fe3f1b32997464080d6456a66dd2cb5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_de7237cc0a6b47fcbc3ae22b6db02b8b",
            "placeholder": "​",
            "style": "IPY_MODEL_10f81f2778c84d918796329d855b99a0",
            "value": " 2743/2743 [00:01&lt;00:00, 1634.28ex/s]"
          }
        },
        "291af4a7e0254ccea17b9fa0827ff300": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b359af007c8849d4a8a0187bedd4ab2c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "16e48341d7724f4f927b978c745108a5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "6da1d8566ac441cdadbb8b211b9b164a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cc70cfe677124a4d9df99c0fc327ff44": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "de7237cc0a6b47fcbc3ae22b6db02b8b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "10f81f2778c84d918796329d855b99a0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4593b4744ab94b20bf988c06dc926628": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_15c90f3f94e346a3be88e7c1a0a2da58",
              "IPY_MODEL_05967551ce224aa4b11b197364bb71e8",
              "IPY_MODEL_bca951e1f3194ff19b2885238e0e453d"
            ],
            "layout": "IPY_MODEL_d96c9cf6d49f431c9fbeaf5fa342b138"
          }
        },
        "15c90f3f94e346a3be88e7c1a0a2da58": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d7ecd06b99d2435283f0751314e3146a",
            "placeholder": "​",
            "style": "IPY_MODEL_80419402cd0e4a9b9449a71a9a25adfb",
            "value": "100%"
          }
        },
        "05967551ce224aa4b11b197364bb71e8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_41e09c4f36f749e6b27ab9d14f4ce37f",
            "max": 343,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_2748a0b464cf4a48a7e2cb8504b34b46",
            "value": 343
          }
        },
        "bca951e1f3194ff19b2885238e0e453d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_bd05e0ba4aa343529c57559358c4dde6",
            "placeholder": "​",
            "style": "IPY_MODEL_bfbac8f75b7846629fb4d0e34d2756bb",
            "value": " 343/343 [00:00&lt;00:00, 1636.25ex/s]"
          }
        },
        "d96c9cf6d49f431c9fbeaf5fa342b138": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d7ecd06b99d2435283f0751314e3146a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "80419402cd0e4a9b9449a71a9a25adfb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "41e09c4f36f749e6b27ab9d14f4ce37f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "2748a0b464cf4a48a7e2cb8504b34b46": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "bd05e0ba4aa343529c57559358c4dde6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bfbac8f75b7846629fb4d0e34d2756bb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "57f73569d7bf43caaa746bda9df0af42": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_8f885c7148bd4df3a4b0bce9e4078bf2",
              "IPY_MODEL_c10c70680c6346d5bc7699b9022a122a",
              "IPY_MODEL_0f642d39819c4500a7e9700b6abbf89d"
            ],
            "layout": "IPY_MODEL_294743a2749e4eccb5eecdff89682d9e"
          }
        },
        "8f885c7148bd4df3a4b0bce9e4078bf2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_087901849abb4ecd8d2fbe52b3cec88d",
            "placeholder": "​",
            "style": "IPY_MODEL_792fa0940156469bb2b2154cdd9c51fa",
            "value": "100%"
          }
        },
        "c10c70680c6346d5bc7699b9022a122a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2101e89a3b2042aa85d719fa569b9681",
            "max": 343,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_4957c647784d495083fc6df9094db54f",
            "value": 343
          }
        },
        "0f642d39819c4500a7e9700b6abbf89d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_447aeed254a04383bf739a1637619394",
            "placeholder": "​",
            "style": "IPY_MODEL_fcb3a85c62b54ad89985c556ee83d7f5",
            "value": " 343/343 [00:00&lt;00:00, 1436.64ex/s]"
          }
        },
        "294743a2749e4eccb5eecdff89682d9e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "087901849abb4ecd8d2fbe52b3cec88d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "792fa0940156469bb2b2154cdd9c51fa": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "2101e89a3b2042aa85d719fa569b9681": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4957c647784d495083fc6df9094db54f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "447aeed254a04383bf739a1637619394": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "fcb3a85c62b54ad89985c556ee83d7f5": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "747c02f9d9254b09ba811633440fba37": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_6ec322204cef4463b88c75fd850d0d50",
              "IPY_MODEL_bd84a09a2be84f6fb732c73f699d70f8",
              "IPY_MODEL_e6545999e0634369992fb30021ab5e9c"
            ],
            "layout": "IPY_MODEL_f46c70c3f68947bb8f3249850ca2c3fe"
          }
        },
        "6ec322204cef4463b88c75fd850d0d50": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e02689de93844bb2b13934cd6e7106f6",
            "placeholder": "​",
            "style": "IPY_MODEL_448bf4d5f9894dd6927b72a5d4e859de",
            "value": "100%"
          }
        },
        "bd84a09a2be84f6fb732c73f699d70f8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e65a4ba44fec4506bc6001bdfc9930a0",
            "max": 3,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f581d78126b34ea7b673bbf835175e35",
            "value": 3
          }
        },
        "e6545999e0634369992fb30021ab5e9c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a6edf96ffb24434eae98dfb74f2c83e1",
            "placeholder": "​",
            "style": "IPY_MODEL_4c41ba8e69da487e820fbe58b85f6849",
            "value": " 3/3 [00:00&lt;00:00, 72.84ex/s]"
          }
        },
        "f46c70c3f68947bb8f3249850ca2c3fe": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e02689de93844bb2b13934cd6e7106f6": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "448bf4d5f9894dd6927b72a5d4e859de": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e65a4ba44fec4506bc6001bdfc9930a0": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f581d78126b34ea7b673bbf835175e35": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a6edf96ffb24434eae98dfb74f2c83e1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4c41ba8e69da487e820fbe58b85f6849": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    },
    "accelerator": "GPU"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}