hf_trainer_rnn.ipynb

{
  "cells": [
    {
      "cell_type": "markdown",
      "metadata": {
        "id": "view-in-github",
        "colab_type": "text"
      },
      "source": [
        "<a href=\"https://colab.research.google.com/github/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/hf_trainer_rnn.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Setup\n",
        "\n",
        "First, set up the required Python modules and perform some general configuration.\n",
        "\n",
        "(This part of the code follows the [CNN notebook](https://github.com/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/hf_trainer_cnn.ipynb) that you should already be familiar with.)"
      ],
      "metadata": {
        "id": "wo13ZXoZYB6J"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Install the required Python packages using [pip](https://en.wikipedia.org/wiki/Pip):\n",
        "\n",
        "* [`transformers`](https://huggingface.co/docs/transformers/index) is a popular deep learning package\n",
        "* [`datasets`](https://huggingface.co/docs/datasets/) provides support for loading, creating, and manipulating datasets\n",
        "* [`evaluate`](https://huggingface.co/docs/evaluate/index) is a library for easily evaluating machine learning models and datasets\n",
        "* [`accelerate`](https://pypi.org/project/accelerate/) is a wrapper we need to install in order to train torch models using a transformers trainer\n",
        "\n",
        "Both `transformers` and `datasets` are used extensively on this course."
      ],
      "metadata": {
        "id": "qKDDc9isYnxz"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "!pip3 install -q transformers datasets evaluate accelerate"
      ],
      "metadata": {
        "id": "4pquj9Xoxaza",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "46aec08c-c884-4d2f-dd67-c26caaf3be2a"
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m527.3/527.3 kB\u001b[0m \u001b[31m12.4 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m84.1/84.1 kB\u001b[0m \u001b[31m3.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m116.3/116.3 kB\u001b[0m \u001b[31m6.7 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m39.9/39.9 MB\u001b[0m \u001b[31m13.8 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m134.8/134.8 kB\u001b[0m \u001b[31m4.6 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[2K   \u001b[90m━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━━\u001b[0m \u001b[32m194.1/194.1 kB\u001b[0m \u001b[31m1.3 MB/s\u001b[0m eta \u001b[36m0:00:00\u001b[0m\n",
            "\u001b[?25h\u001b[31mERROR: pip's dependency resolver does not currently take into account all the packages that are installed. This behaviour is the source of the following dependency conflicts.\n",
            "cudf-cu12 24.4.1 requires pyarrow<15.0.0a0,>=14.0.1, but you have pyarrow 17.0.0 which is incompatible.\n",
            "ibis-framework 8.0.0 requires pyarrow<16,>=2, but you have pyarrow 17.0.0 which is incompatible.\u001b[0m\u001b[31m\n",
            "\u001b[0m"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "(Above, the `!` at the start of the line tells the notebook to run the line as an operating system command rather than Python code, and the `-q` argument to `pip` runs the command in \"quiet\" mode, with less output.)"
      ],
      "metadata": {
        "id": "JyyZCbB0Yxmq"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We'll also use the [`pprint`](https://docs.python.org/3/library/pprint.html) (\"pretty-print\") module to format output more readably below. The only difference to just using `print` is that some data structures will be easier to read and interpret."
      ],
      "metadata": {
        "id": "N2oLE1FpY09U"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from pprint import PrettyPrinter\n",
        "\n",
        "pprint = PrettyPrinter(compact=True).pprint"
      ],
      "metadata": {
        "id": "cQ63zw6BY7tn"
      },
      "execution_count": 2,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Finally, we will reduce logging output. The `transformers` library by default produces fairly verbose logging. Commenting out the following code will enable low-priority output (`INFO` logging level and below)."
      ],
      "metadata": {
        "id": "xkrZ6e_FY_2e"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import logging\n",
        "\n",
        "logging.disable(logging.INFO)"
      ],
      "metadata": {
        "id": "ZH2bY_GjZG24"
      },
      "execution_count": 3,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---\n",
        "\n",
        "# Download and prepare data\n",
        "\n",
        "We will again use the `datasets` library function [`load_dataset`](https://huggingface.co/docs/datasets/master/en/package_reference/loading_methods#datasets.load_dataset) to load a dataset for our experiments."
      ],
      "metadata": {
        "id": "W4x7GbT2ZKUJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import datasets\n",
        "\n",
        "\n",
        "dataset = datasets.load_dataset(\"imdb\")"
      ],
      "metadata": {
        "id": "5DKskTuoyCf-",
        "outputId": "19c458e6-f425-4673-9aeb-e1a336647f3d",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 368,
          "referenced_widgets": [
            "601cc593e1c3455295499c1dd060cc84",
            "445bff27d27e4852ae7b4124afd2e61d",
            "dd9a38f5807d42f992fbed94e8d05670",
            "35d2bc508fd04416a2c74f95704e6c19",
            "2d06c2d7ec1d478cb173f5b2945e5893",
            "527035e8d4474901a30e3e280f8678a5",
            "8dbb4c21506d4e758fb47c4faf2057b6",
            "c152ffd4292a401398ff2e9cdf29bc62",
            "0febb51b098144438b6864cef73596bc",
            "67d5b1f99c6d4ed399a582ec282eb481",
            "b856be75aefc46bc881bc820330e9678",
            "3f7a4384cdf94392a233e2ddaa2fbe8c",
            "17de7424a0914ba980c52fd1474fa3cc",
            "29330125b3dc4c61a6a935cb4e999220",
            "06c4cc10525c4282980a5092180790c0",
            "83cd8fe2e6574af18f832c7df2708279",
            "3fed8e4d58744a8898636e4e99b94c09",
            "18b35cc54dd945b5b216f8be5bf581b8",
            "e63fd4ccf25740939cc67a5479f89ec9",
            "37ae6bc371164ddbac7492ca4ce6cc92",
            "132d7d5b41d74c708c52cf8ef66c199b",
            "f7400c26a9c04b6ebda65d72b82ba4b7",
            "e7a05a28271f4507afe255597b845c77",
            "ae0e7e708e8047d7a2b3062d4e503a2c",
            "c51bc41feca04969906bf87f556a61f6",
            "33531cd5b4b24ad59c6c8747957a9a5d",
            "3dc245576759495aa9605bbe78d09359",
            "6d18012122124723add21fed1dc1b83f",
            "d3bb189c23da4f56aff2593e9a430a2f",
            "1ddd531895e248c8ab6d1338fc246913",
            "9e5470e390b34b1ca8516d73ae42a773",
            "3f3df2b09c694dfba1b7d2eb96a0cb0f",
            "9f52edc6bdc44dd7993dc704a0e8bc83",
            "cb54f9f6f972425b89fb6ee19a63b1ba",
            "6cbbab0ec97d4cf29f3299f4d86d942a",
            "30b52d610cab4090a7ea704f2d94a7a8",
            "cb32d657f1d54d2fa71ce28655d50ea9",
            "a4b7bdc66e234a37bfe85bb76266bdd2",
            "41f7feb9fae040cba47e0974a8264ec4",
            "ebe2001655194662b20ee5c1c31a1f63",
            "559a05a264ec478299b08884847d9f47",
            "4cefa0cfde054ba0b6e08fcd2b86fb62",
            "10845084974c47f0bc80c5d9fd5e39eb",
            "034010e0a32f4a6fab9638ca65eaa69e",
            "beaaaf5eea884de690056f7bc0db35ad",
            "d78ddbb4a9f44931804dc2404072207a",
            "63b41c445ccc4369af8b3c62d10caa27",
            "a6be26f9d0b34cf8a3943e473afe6286",
            "c68081a71c5e41eca1c0185ea6b234b5",
            "361a55924ed046238b5022b7b285f1b1",
            "be1be61b1deb4cdb888f19df982f70a6",
            "4ed0dafa0f894de1bab254e5781f7085",
            "967a4a82f8bb4014ac8a65712670db4c",
            "ea4d3cbe6dd247f6bce1b9032f03e47d",
            "997ade0ebda24450afc0fcd37d319581",
            "f69c549312234802b6ed88b1469075e3",
            "3bb78037136d49b1b0d66718b63b359a",
            "957c896524b9431393e1dca0d9fdcabb",
            "ed4ea35ff9394cb3a29a50a41848270c",
            "545b739cc7eb47e39f637e84a516ad3b",
            "367a65e921064433b1d85a05c2c364aa",
            "8b79020824864c7b9a9374cd75986b89",
            "e68e65143753434ca61778c1342356d7",
            "b717d2a5bc8a49459c9147a17d8794dc",
            "a52638aafb934b60b982203c4061e24e",
            "8b2d730dc0e14ef680de56a648d13aeb",
            "d18d65a927494eb5a567abdeb71a7f93",
            "acbe022ef92042d08e523e66571a68a1",
            "1d8a01207976474e8b5b2e8189839929",
            "b812780654c743c5b6ea1783a80b0929",
            "a5d11c40fb294ec8bebe8742c9991022",
            "5f866679aaa042c89358c609533c9ade",
            "25beb6fb3dda462fb308d030f0051700",
            "5b8b1e48d0154d5587d5b0dd92307309",
            "246a45f1e8b24b369a0b27ba035fe5ff",
            "38b640e024b24fd79349d40699ddc95d",
            "bd38d55fd0714892aeefd06f08e91303"
          ]
        }
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/huggingface_hub/utils/_token.py:89: UserWarning: \n",
            "The secret `HF_TOKEN` does not exist in your Colab secrets.\n",
            "To authenticate with the Hugging Face Hub, create a token in your settings tab (https://huggingface.co/settings/tokens), set it as secret in your Google Colab and restart your session.\n",
            "You will be able to reuse this secret in all of your notebooks.\n",
            "Please note that authentication is recommended but still optional to access public models or datasets.\n",
            "  warnings.warn(\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading readme:   0%|          | 0.00/7.81k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "601cc593e1c3455295499c1dd060cc84"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading data:   0%|          | 0.00/21.0M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "3f7a4384cdf94392a233e2ddaa2fbe8c"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading data:   0%|          | 0.00/20.5M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "e7a05a28271f4507afe255597b845c77"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading data:   0%|          | 0.00/42.0M [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "cb54f9f6f972425b89fb6ee19a63b1ba"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Generating train split:   0%|          | 0/25000 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "beaaaf5eea884de690056f7bc0db35ad"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Generating test split:   0%|          | 0/25000 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "f69c549312234802b6ed88b1469075e3"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Generating unsupervised split:   0%|          | 0/50000 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "d18d65a927494eb5a567abdeb71a7f93"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's see what the dataset contains:"
      ],
      "metadata": {
        "id": "FVo6_CS2eeIq"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "print(dataset)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "y3K4Fkz7yce7",
        "outputId": "d968ef48-5108-4801-9bb3-d50672b77e20"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "DatasetDict({\n",
            "    train: Dataset({\n",
            "        features: ['text', 'label'],\n",
            "        num_rows: 25000\n",
            "    })\n",
            "    test: Dataset({\n",
            "        features: ['text', 'label'],\n",
            "        num_rows: 25000\n",
            "    })\n",
            "    unsupervised: Dataset({\n",
            "        features: ['text', 'label'],\n",
            "        num_rows: 50000\n",
            "    })\n",
            "})\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "and print out an example:"
      ],
      "metadata": {
        "id": "o17ZWi_kf4AJ"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "dataset = dataset.shuffle()\n",
        "del dataset[\"unsupervised\"]\n",
        "pprint(dataset[\"train\"][0])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Wd__TIUnf9PD",
        "outputId": "79843140-14d0-432b-dabb-da0307bc4593"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'label': 0,\n",
            " 'text': \"In case half of this film's footage looks strangely familiar, it \"\n",
            "         'means you watch way too much of this gory Italian cult-crap! For you '\n",
            "         'see, the notorious demigod Lucio Fulci did not only produce this '\n",
            "         'movie, he also took the liberty of re-using the most sadistic '\n",
            "         'killing sequences in his own (and more easily traceable) \"Cat in the '\n",
            "         'Brain\". The opening scene already, in which a prostitute is brutally '\n",
            "         \"decapitated with an axe, features in Fulci's later film and so do \"\n",
            "         \"another handful of killings and sleaze moments. Andrea Bianchi's \"\n",
            "         '\"Masssacre\" fails miserably as a giallo, since the search for the '\n",
            "         'sadistic killer among a movie-crew shooting a horror movie is all '\n",
            "         \"but compelling and suspenseful, but it's still good entertainment if \"\n",
            "         \"you're into cheap 'n shlocky horror trash. The incoherent script \"\n",
            "         \"introduces a whole bunch of repugnant characters who're all \"\n",
            "         'potential maniacs, but none of the red herrings Biachi comes up with '\n",
            "         'are plausible and you can point out the killer almost immediately. '\n",
            "         'Whatever remains to enjoy are the truly misogynous make-up effects '\n",
            "         'and the hilariously awful acting performances of the ensemble cast. '\n",
            "         \"Whenever there isn't any bloodshed on screen, like between the first \"\n",
            "         'and second murder, \"Massacre\" is a slow and almost intolerable with '\n",
            "         'its inane dialogs and thoroughly unexciting photography. Thankfully '\n",
            "         'in the second half, there are women getting impaled on fences and '\n",
            "         'males being stabbed repeatedly with rusty spikes. The music is crap '\n",
            "         'and the use of filming locations is very unimaginative. My advise '\n",
            "         'would be to skip this puppy and go straight for the aforementioned '\n",
            "         '\"Cat in the Brain\". That one features ALL the great moments of '\n",
            "         '\"Massacre\", and then some.'}\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "---\n",
        "\n",
        "# Tokenize and vectorize data\n",
        "\n",
        "(This part of the code follows the [CNN notebook](https://github.com/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/hf_trainer_cnn.ipynb) that you should already be familiar with.)"
      ],
      "metadata": {
        "id": "8KF9UtzUbrBA"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "To tokenize and vectorize the texts of our dataset, we will again use previously created tokenizers through the simple [`AutoTokenizer`](https://huggingface.co/docs/transformers/model_doc/auto#transformers.AutoTokenizer) class.\n",
        "\n",
        "The [`AutoTokenizer.from_pretrained`](https://huggingface.co/docs/transformers/model_doc/auto#transformers.AutoTokenizer.from_pretrained) function can load the tokenizer associated with any of the large number of models found in the [Hugging Face models repository](https://huggingface.co/models). Here, our texts are in English, and we'll load the tokenizer for the [`bert-base-cased`](https://huggingface.co/bert-base-cased) model.\n",
        "\n",
        "(**Note**: we're not actually using the BERT model here, just its tokenizer.)"
      ],
      "metadata": {
        "id": "H9RBff6Rkfyt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import transformers\n",
        "\n",
        "model_name = \"bert-base-cased\"\n",
        "tokenizer = transformers.AutoTokenizer.from_pretrained(model_name)"
      ],
      "metadata": {
        "id": "wjrAGcFtymJF",
        "outputId": "a293935d-10ab-43ac-d335-da745e7390f1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 201,
          "referenced_widgets": [
            "1a2e80a6758447719a574f0b30741487",
            "d09a31f1abc949f1b5630c2b225750f9",
            "c288a59070c048f69a20e59ae49947a9",
            "f8d20844505046368fbb6bff2ac00cb8",
            "c86cdac9336d468aa74f69642244482b",
            "240339f70ee6435bbd5efc6accf66863",
            "0ac4f9a91a78497389cbe3002d3f0924",
            "7b1ebe4cf37940b5b7666f653b9d88df",
            "e53ab6a12d774a66aa8ac5da4b96f714",
            "72382cc803174b3da35596eee5e8a3cd",
            "dd9640a5488e44a69b94d2d5770f4f2e",
            "0f70760177464f55b77db49629f8b4be",
            "fc96b4b213064fd5b8f096b4ff117357",
            "97173b8da10e4312b7b419d448fe27a3",
            "4003709e9752457ea2ad3e5aeb86543f",
            "ca4c31ccdc6d4d2a96bc5bfe3bdd6c70",
            "9459564b479149cab3e45a05ed52360e",
            "0afed6b2c1024f199cacfdedb8cedd03",
            "b04bf0fbca71412595b03d96d6bd210d",
            "f67a7ac769cf43a18fbb0445654101ff",
            "33b11d1b526142f9a3a45e198847a33e",
            "99c7b885e0e949b584d27e9c866faf06",
            "9d27c45cffad47cdb1a5788e245b8aef",
            "7b98864f4e824a6db2022a25a7597c2c",
            "a428356220f94d069f2129de8a4824b8",
            "d3bc3fa81af5424482b3a797ad198797",
            "da7e4f5bb7854eaea4c7e9594ca03399",
            "44b1a777f5794986a17fe474578b1373",
            "b8ce620748fd43f988948711220518b1",
            "9595357778854e0eb6f6468737e2b396",
            "dc9d236e4f5c45428285bb9eb3668598",
            "a63d3bdc7830487bac7fd5c11e0418d4",
            "47ec1edee0d34965944cf295c0aa1d13",
            "d45fb6b1db644a6f90626069345b452f",
            "e23a7f66f25e43069e58662f37e1a397",
            "84abe125a60542c0a5f664bd3e891172",
            "84186a90796049c998ba69b8dc3b1004",
            "4eeb94d9f3c64808b4b82feecdd1fe26",
            "8a54a001c2124c0694e4981350685da5",
            "eb5fef2ec0564d0782025d29db163e8f",
            "d0dda846da934384a2ee06bc140543bf",
            "00b4f767f0e64004be48768f7bc34fd3",
            "1210f69cbb68400e8474ba10046be85b",
            "07c4b00ae6aa45c9b7e3036e66bd5588"
          ]
        }
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "tokenizer_config.json:   0%|          | 0.00/49.0 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "1a2e80a6758447719a574f0b30741487"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "config.json:   0%|          | 0.00/570 [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "0f70760177464f55b77db49629f8b4be"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "vocab.txt:   0%|          | 0.00/213k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "9d27c45cffad47cdb1a5788e245b8aef"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "tokenizer.json:   0%|          | 0.00/436k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "d45fb6b1db644a6f90626069345b452f"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/transformers/tokenization_utils_base.py:1601: FutureWarning: `clean_up_tokenization_spaces` was not set. It will be set to `True` by default. This behavior will be depracted in transformers v4.45, and will be then set to `False` by default. For more details check this issue: https://github.com/huggingface/transformers/issues/31884\n",
            "  warnings.warn(\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "As in the [CNN notebook](https://github.com/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/hf_trainer_cnn.ipynb), we will define a simple tokenization function and tokenize and vectorize our whole dataset with the tokenizer by calling the [`Dataset.map`](https://huggingface.co/docs/datasets/v2.14.4/en/package_reference/main_classes#datasets.Dataset.map) function.\n",
        "\n",
        "Note that here we're providing a `max_length` argument and `truncation=True` in the tokenizer call. This limits the maximum length of outputs to the given length (see the [tokenizers documentation](https://huggingface.co/docs/transformers/preprocessing#everything-you-always-wanted-to-know-about-padding-and-truncation) for details). This makes training faster, potentially at some cost in performance."
      ],
      "metadata": {
        "id": "mjEeyPsXc12X"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Define a simple function that applies the tokenizer\n",
        "def tokenize(example):\n",
        "    return tokenizer(\n",
        "        example[\"text\"],\n",
        "        max_length=128,\n",
        "        truncation=True,\n",
        "    )\n",
        "\n",
        "# Apply the tokenizer to the whole dataset using .map()\n",
        "dataset = dataset.map(tokenize)"
      ],
      "metadata": {
        "id": "K2c4lFKTywlu",
        "outputId": "7beda044-f1d7-4b3b-bd87-64429693ccba",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 101,
          "referenced_widgets": [
            "834bf849fbbb421fb69f848a1e4f4080",
            "102a88efa1ea4693a096482cacd54727",
            "02bc83cc4d0c4a6facad8ebab1349d01",
            "87d0828eac384e459355cc292eacd519",
            "73ec6b13c83744b99ce7bbbb5c01385f",
            "af55a9f236c94b4fadef6a02829a387f",
            "11743d1ce51245708e60dc6e04bfef71",
            "508d1dd5624048cb90a75e7f8b088037",
            "560039c590614a3da97e3ee1384106c4",
            "2316c6176a5d40658a3afa8962914e51",
            "ddb780f98876493eaf26d7ed344e4330",
            "91a865277aa4458eb6e872895f698f41",
            "38f87798046f4ee68f301e51cf329cb6",
            "cccc86f8fc504edd948d683b07a685c2",
            "f0977ede73fe4bf3848ac3bf3f1b6403",
            "ce5602958e5a45ca91076294c73aaafb",
            "3d38c3b8932b46088c3b4d762c3291a7",
            "193c6066997549e7b48c8d9842b352b6",
            "546e890771044b1d9f245856ca199f7b",
            "4b8cb267340248d281eccf3d75f02738",
            "4f7dd067bf234ee0b136d7c0245710a8",
            "5518613d33944dc7a50f1fc0c8797109"
          ]
        }
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map:   0%|          | 0/25000 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "834bf849fbbb421fb69f848a1e4f4080"
            }
          },
          "metadata": {}
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Map:   0%|          | 0/25000 [00:00<?, ? examples/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "91a865277aa4458eb6e872895f698f41"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "---\n",
        "\n",
        "# Build model\n",
        "\n",
        "As usual, we will create a PyTorch model class with an `__init__()` function that creates the layers and a `forward()` function which implements the actual computation. For more information on these, please see the [PyTorch tutorial](https://pytorch.org/tutorials/beginner/introyt/modelsyt_tutorial.html)."
      ],
      "metadata": {
        "id": "AiX7WeW2X5gv"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We're here building a simple RNN with the following structure:\n",
        "\n",
        "* As in the [CNN](https://github.com/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/hf_trainer_cnn.ipynb), the token IDs are first mapped to embeddings of a user-specified size (`config.embedding_dim`) in a [torch.nn.Embedding](https://pytorch.org/docs/stable/generated/torch.nn.Embedding.html) layer. Note that here the embeddings are initialized randomly and learned along with other model weights. In real-world applications, the embeddings would typically be initialized with previously learned weights.\n",
        "* Second, the embedded inputs are passed through an RNN ([torch.nn.RNN](https://pytorch.org/docs/stable/generated/torch.nn.RNN.html)), which produces a series of outputs ($(y_1, \\ldots, y_n)$, where $n$ is the length of the input) and the final hidden state $h_n$. Here, we will only use the last output $y_n$.\n",
        "* Finally, there is a fully connected layer ([torch.nn.Linear](https://pytorch.org/docs/stable/generated/torch.nn.Linear.html)) that maps the last RNN output to the two possible output values of the classifier.\n",
        "\n",
        "We can interpret this model as processing the input step by step, attempting to identify tokens (embeddings) that in the context of its previous input express either positive or negative opinions, and to output a value at the end of the sequence that can be mapped to the positive or negative class."
      ],
      "metadata": {
        "id": "8UQlKM1Vfrha"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "In the `forward` function we mostly just pass the input through the layers, with the following additional steps:\n",
        "\n",
        "* To invoke the RNN, we need to provide the value of the initial hidden state $h_0$. Here we simply use [torch.zeros](https://pytorch.org/docs/stable/generated/torch.zeros.html) to create a tensor of the appropriate size filled with zeros.\n",
        "* To get the value of the last item in the sequence of RNN outputs (`rnn_outputs`), we slice the three-dimensional tensor (batch, rnn step, output dim) with `rnn_outputs[:, -1, :]`. This returns all values in the first and last dimensions, and the last in the second. If you are not familar with this syntax, consider the following example:"
      ],
      "metadata": {
        "id": "mZqIlxnDOxbj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import numpy\n",
        "\n",
        "a = numpy.array([\n",
        "  [[11], [12], [13]],\n",
        "  [[21], [22], [23]],\n",
        "  [[31], [32], [33]],\n",
        "])\n",
        "\n",
        "print(a[:, -1, :])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "IPWGCx8ODwS9",
        "outputId": "3e0b237e-68b1-4283-8652-dd943cd8e9ad"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[[13]\n",
            " [23]\n",
            " [33]]\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "(The two points above can be considered technical details and understanding them in detail is not required to understand the model.)"
      ],
      "metadata": {
        "id": "JlGCWXiSEvH8"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Here's the model:"
      ],
      "metadata": {
        "id": "UCgAqPAeEypM"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import torch\n",
        "\n",
        "\n",
        "# This gives a new name to the config class, just for convenience\n",
        "BasicConfig = transformers.PretrainedConfig\n",
        "\n",
        "\n",
        "# This is the model\n",
        "class SimpleRNN(transformers.PreTrainedModel):\n",
        "\n",
        "    config_class = BasicConfig\n",
        "\n",
        "    # In the initialization method, one instantiates the layers\n",
        "    # these will be the parameters of the model\n",
        "    def __init__(self, config):\n",
        "        super().__init__(config)\n",
        "        # Embedding layer: vocab size x embedding dim\n",
        "        self.embeddings = torch.nn.Embedding(\n",
        "            num_embeddings=config.vocab_size,\n",
        "            embedding_dim=config.embedding_dim\n",
        "        )\n",
        "        # RNN with configurable hidden size and nonlinearity\n",
        "        self.rnn = torch.nn.RNN(\n",
        "            input_size=config.embedding_dim,\n",
        "            hidden_size=config.hidden_size,\n",
        "            num_layers=config.num_layers,\n",
        "            nonlinearity=config.nonlinearity,\n",
        "            batch_first=True\n",
        "        )\n",
        "        # Output layer: embedding size to output size\n",
        "        self.output_layer = torch.nn.Linear(\n",
        "            in_features=config.hidden_size,\n",
        "            out_features=config.num_labels\n",
        "        )\n",
        "        # Loss function: standard loss for classification\n",
        "        self.loss = torch.nn.CrossEntropyLoss()\n",
        "\n",
        "    def forward(self, input_ids, labels=None, attention_mask=None):\n",
        "        # Embed input ids\n",
        "        x = self.embeddings(input_ids)\n",
        "        # Set initial hidden state to all-zero values\n",
        "        batch_size = x.shape[0]\n",
        "        h0 = torch.zeros(\n",
        "            (self.config.num_layers, batch_size, self.config.hidden_size),\n",
        "            device=input_ids.device    # place on same device as input\n",
        "        )\n",
        "        # Run RNN repeatedly to get sequence of outputs and last hidden state\n",
        "        rnn_outputs, h_n = self.rnn(x, h0)\n",
        "        # Get last RNN output\n",
        "        y_n = rnn_outputs[:, -1, :]\n",
        "        # Map to outputs with fully connected layer\n",
        "        output = self.output_layer(y_n)\n",
        "\n",
        "        # Return value computed as in MLP and CNN:\n",
        "        if labels is not None:\n",
        "            # We have labels, so we can calculate the loss\n",
        "            return (self.loss(output,labels), output)\n",
        "        else:\n",
        "            # No labels, so just return the output\n",
        "            return (output,)"
      ],
      "metadata": {
        "id": "9kyQQhu0_wep"
      },
      "execution_count": 10,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "---\n",
        "\n",
        "# Configure and train model"
      ],
      "metadata": {
        "id": "Tmj91CdCT-fJ"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "We'll first configure and instantiate the model. Here `vocab_size` should always be the vocabulary size of the tokenizer and `num_labels` the number of unique labels in the data (as here), but the others are hyperparameters that you can choose:\n",
        "\n",
        "* `embedding_dim`: the size of the word (i.e. token) embeddings\n",
        "* `hidden_size`: the size of the RNN hidden state vector _h_\n",
        "* `num_layers`: number of stacked RNN layers\n",
        "* `nonlinearity`: the non-linear function to apply in RNN (`'tanh'` or `'relu'`)"
      ],
      "metadata": {
        "id": "pVTopuNPW26S"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "config = BasicConfig(\n",
        "    vocab_size = tokenizer.vocab_size,\n",
        "    num_labels = len(set(dataset[\"train\"][\"label\"])),\n",
        "    embedding_dim = 64,\n",
        "    hidden_size = 96,\n",
        "    num_layers = 1,\n",
        "    nonlinearity = \"tanh\",\n",
        ")\n",
        "\n",
        "model = SimpleRNN(config)"
      ],
      "metadata": {
        "id": "nd_usta7WzTX"
      },
      "execution_count": 11,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "Training arguments are set similarly as in the [CNN notebook](https://github.com/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/hf_trainer_cnn.ipynb). Many number of these settings relate to the frequency of evaluation and output during training, but the following are hyperparameters that you may wish to adjust:\n",
        "\n",
        "* `learning_rate`: the step size for weight updates\n",
        "* `per_device_train_batch_size`: number of examples per batch\n",
        "* `max_steps`: the maximum number of steps to train for"
      ],
      "metadata": {
        "id": "q5xkBXWDUL-_"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Set training arguments\n",
        "trainer_args = transformers.TrainingArguments(\n",
        "    \"checkpoints\",\n",
        "    evaluation_strategy=\"steps\",\n",
        "    logging_strategy=\"steps\",\n",
        "    load_best_model_at_end=True,\n",
        "    eval_steps=500,\n",
        "    logging_steps=500,\n",
        "    learning_rate=0.001,\n",
        "    per_device_train_batch_size=8,\n",
        "    max_steps=2500,\n",
        ")"
      ],
      "metadata": {
        "id": "gVW-yGhYItWk",
        "outputId": "c7674fac-86cb-411d-d670-38c4c9699408",
        "colab": {
          "base_uri": "https://localhost:8080/"
        }
      },
      "execution_count": 12,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.10/dist-packages/transformers/training_args.py:1525: FutureWarning: `evaluation_strategy` is deprecated and will be removed in version 4.46 of 🤗 Transformers. Use `eval_strategy` instead\n",
            "  warnings.warn(\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "We'll then define the standard accuracy metric (ratio of correct out of all predictions), create a [DataCollatorWithPadding](https://huggingface.co/docs/transformers/main_classes/data_collator#transformers.DataCollatorWithPadding) to pad inputs to the same length (as required for batching) and an [EarlyStoppingCallback](https://huggingface.co/docs/transformers/main_classes/callback#transformers.EarlyStoppingCallback) to stop training when performance fails to improve for the given number of evaluations.\n",
        "\n",
        "(These should all be familiar to you from the [CNN notebook](https://github.com/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/hf_trainer_cnn.ipynb))"
      ],
      "metadata": {
        "id": "lRE6YtHvUTcj"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "import evaluate\n",
        "accuracy = evaluate.load(\"accuracy\")\n",
        "\n",
        "\n",
        "def compute_accuracy(outputs_and_labels):\n",
        "    outputs, labels = outputs_and_labels\n",
        "    predictions = outputs.argmax(axis=-1) #pick the index of the \"winning\" label\n",
        "    return accuracy.compute(predictions=predictions, references=labels)\n",
        "\n",
        "data_collator = transformers.DataCollatorWithPadding(tokenizer)\n",
        "\n",
        "# Argument gives the number of steps of patience before early stopping\n",
        "early_stopping = transformers.EarlyStoppingCallback(\n",
        "    early_stopping_patience=5\n",
        ")"
      ],
      "metadata": {
        "id": "miilVFfCIhR1",
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 69,
          "referenced_widgets": [
            "154569f9633046af8a4bae96088b832c",
            "63aede7845044c29ad2ec0436339a177",
            "7a0bc7df773048ce91b1697044a84118",
            "81ecb2b2d8d340f98ad5c363a758e42a",
            "7b3b0424b1a449aa82010f96c16d213a",
            "7ab3d996813446398b7de30bb3c88f36",
            "8de4b5f9b2374eee99a99a110480401d",
            "9fb2dabeecee4172985b38f93163053c",
            "cbb4fe0681a84febbe5109d237097b06",
            "9207061018444358875b76f16560dc8f",
            "bd71b5eea5414cf69a85b58dcd8689ac"
          ]
        },
        "outputId": "7e39b582-75c5-4a65-95df-127dbc900f18"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "Downloading builder script:   0%|          | 0.00/4.20k [00:00<?, ?B/s]"
            ],
            "application/vnd.jupyter.widget-view+json": {
              "version_major": 2,
              "version_minor": 0,
              "model_id": "154569f9633046af8a4bae96088b832c"
            }
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Finally, as in the [CNN notebook](https://github.com/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/hf_trainer_cnn.ipynb), we'll create a simple custom [callback](https://huggingface.co/docs/transformers/main_classes/callback) to store values logged during training so that we can more easily examine them later. (This is only needed for visualization and is not necessary to understand in detail.)"
      ],
      "metadata": {
        "id": "pd3o8RIdaWqb"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "from collections import defaultdict\n",
        "\n",
        "class LogSavingCallback(transformers.TrainerCallback):\n",
        "    def on_train_begin(self, *args, **kwargs):\n",
        "        self.logs = defaultdict(list)\n",
        "        self.training = True\n",
        "\n",
        "    def on_train_end(self, *args, **kwargs):\n",
        "        self.training = False\n",
        "\n",
        "    def on_log(self, args, state, control, logs, model=None, **kwargs):\n",
        "        if self.training:\n",
        "            for k, v in logs.items():\n",
        "                if k != \"epoch\" or v not in self.logs[k]:\n",
        "                    self.logs[k].append(v)\n",
        "\n",
        "training_logs = LogSavingCallback()"
      ],
      "metadata": {
        "id": "0EFQebkDYXES"
      },
      "execution_count": 14,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "We then pass the model, trainer arguments, training and evaluation data, metric, the collator, and the callbacks to a [Trainer](https://huggingface.co/docs/transformers/main_classes/trainer) and call `.train()` to train the model."
      ],
      "metadata": {
        "id": "GCuvqB0bVPI2"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "trainer = transformers.Trainer(\n",
        "    model=model,\n",
        "    args=trainer_args,\n",
        "    train_dataset=dataset[\"train\"],\n",
        "    eval_dataset=dataset[\"test\"],\n",
        "    compute_metrics=compute_accuracy,\n",
        "    data_collator=data_collator,\n",
        "    callbacks=[early_stopping, training_logs]\n",
        ")\n",
        "\n",
        "trainer.train()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 305
        },
        "id": "4lKiSRNJVOvJ",
        "outputId": "a815cdcb-434c-4821-aeff-627dd8323419"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "max_steps is given, it will override any value given in num_train_epochs\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='2500' max='2500' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [2500/2500 01:06, Epoch 0/1]\n",
              "    </div>\n",
              "    <table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              " <tr style=\"text-align: left;\">\n",
              "      <th>Step</th>\n",
              "      <th>Training Loss</th>\n",
              "      <th>Validation Loss</th>\n",
              "      <th>Accuracy</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <td>500</td>\n",
              "      <td>0.705200</td>\n",
              "      <td>0.696582</td>\n",
              "      <td>0.514200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1000</td>\n",
              "      <td>0.695100</td>\n",
              "      <td>0.699119</td>\n",
              "      <td>0.510160</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>1500</td>\n",
              "      <td>0.694800</td>\n",
              "      <td>0.694904</td>\n",
              "      <td>0.499600</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2000</td>\n",
              "      <td>0.692800</td>\n",
              "      <td>0.692869</td>\n",
              "      <td>0.509200</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <td>2500</td>\n",
              "      <td>0.690500</td>\n",
              "      <td>0.684632</td>\n",
              "      <td>0.560080</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table><p>"
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "TrainOutput(global_step=2500, training_loss=0.6956649169921875, metrics={'train_runtime': 67.7175, 'train_samples_per_second': 295.345, 'train_steps_per_second': 36.918, 'total_flos': 241858560000.0, 'train_loss': 0.6956649169921875, 'epoch': 0.8})"
            ]
          },
          "metadata": {},
          "execution_count": 15
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "---\n",
        "\n",
        "# Results"
      ],
      "metadata": {
        "id": "VId9dMvAO6aA"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "Evaluate and print out results:"
      ],
      "metadata": {
        "id": "h0VrtFCrO-LO"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "eval_results = trainer.evaluate(dataset[\"test\"])\n",
        "\n",
        "pprint(eval_results)\n",
        "\n",
        "print('Accuracy:', eval_results['eval_accuracy'])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 181
        },
        "id": "7zw3iJ7tPS7V",
        "outputId": "69dbb4d2-31b4-4814-dde8-172484847977"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<IPython.core.display.HTML object>"
            ],
            "text/html": [
              "\n",
              "    <div>\n",
              "      \n",
              "      <progress value='3125' max='3125' style='width:300px; height:20px; vertical-align: middle;'></progress>\n",
              "      [3125/3125 00:09]\n",
              "    </div>\n",
              "    "
            ]
          },
          "metadata": {}
        },
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "{'epoch': 0.8,\n",
            " 'eval_accuracy': 0.56008,\n",
            " 'eval_loss': 0.6846322417259216,\n",
            " 'eval_runtime': 9.9471,\n",
            " 'eval_samples_per_second': 2513.298,\n",
            " 'eval_steps_per_second': 314.162}\n",
            "Accuracy: 0.56008\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "Let's also have a look at training and evaluation loss and evaluation accuracy progression as we did in the [CNN notebook](https://github.com/TurkuNLP/Deep_Learning_in_LangTech_course/blob/master/hf_trainer_cnn.ipynb). (The code here is only for visualization and you do not need to understand it, but you should aim to be able to interpret the plots.)"
      ],
      "metadata": {
        "id": "xm1ZAFTObyf8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "%matplotlib inline\n",
        "import matplotlib.pyplot as plt\n",
        "\n",
        "def plot(logs, keys, labels):\n",
        "    values = sum([logs[k] for k in keys], [])\n",
        "    plt.ylim(max(min(values)-0.1, 0.0), min(max(values)+0.1, 1.0))\n",
        "    for key, label in zip(keys, labels):\n",
        "        plt.plot(logs[\"epoch\"], logs[key], label=label)\n",
        "    plt.legend()\n",
        "    plt.show()\n",
        "\n",
        "plot(training_logs.logs, [\"loss\", \"eval_loss\"], [\"Training loss\", \"Evaluation loss\"])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 450
        },
        "id": "4C2fPl02U9kq",
        "outputId": "be179b49-e3e0-4e5e-823c-2e6f30f47063"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjUAAAGdCAYAAADqsoKGAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAABMHklEQVR4nO3de1xUZeI/8M+ZYS7cUZHh0ijeo8TLorKoW7rhYrZumqUmZbqmpWjGVD+lUry0YqtrbHkrw8uWhV/7WqvJ4gUvfS0KF9ctCzG8hBcGRYKRQeZ6fn8AIyODMiigx8/79TovZ57znOc857xG5+NzznNGEEVRBBEREdFdTtbaHSAiIiK6HRhqiIiISBIYaoiIiEgSGGqIiIhIEhhqiIiISBIYaoiIiEgSGGqIiIhIEhhqiIiISBI8WrsDLcVut+PChQvw9fWFIAit3R0iIiJqBFEUceXKFYSGhkImu/FYzD0Tai5cuACtVtva3SAiIqImOHv2LO67774b1rlnQo2vry+A6pPi5+fXyr0hIiKixjAYDNBqtY7v8Ru5Z0JN7SUnPz8/hhoiIqK7TGNuHeGNwkRERCQJTQo1q1atQnh4ONRqNaKjo5GTk3PD+qmpqejRowc8PT2h1WqRmJiIqqoqt9qsqqpCQkIC2rVrBx8fH4wZMwbFxcVN6T4RERFJkNuhZsuWLdDpdEhOTsaRI0fQu3dvxMXF4eLFiy7rf/LJJ5g7dy6Sk5ORl5eHtLQ0bNmyBa+//rpbbSYmJmLHjh3YunUrDh48iAsXLuCJJ55owiETERGRFAmiKIrubBAdHY3+/ftj5cqVAKqnSmu1WsyaNQtz586tV3/mzJnIy8tDVlaWo+yVV17Bd999h0OHDjWqzfLycrRv3x6ffPIJnnzySQDA8ePHERERgezsbPz2t7+9ab8NBgP8/f1RXl7Oe2qIiNwkiiKsVitsNltrd4UkRi6Xw8PDo8F7Ztz5/nbrRmGz2Yzc3FwkJSU5ymQyGWJjY5Gdne1ym4EDB+Ljjz9GTk4OBgwYgFOnTiEjIwPPPvtso9vMzc2FxWJBbGyso87999+PDh06NBhqTCYTTCaT473BYHDnUImIqIbZbEZRUREqKytbuyskUV5eXggJCYFSqbyldtwKNSUlJbDZbNBoNE7lGo0Gx48fd7nNhAkTUFJSgsGDBzuS/osvvui4/NSYNvV6PZRKJQICAurV0ev1LvebkpKChQsXunN4RER0HbvdjtOnT0MulyM0NBRKpZIPMKXbRhRFmM1mXLp0CadPn0a3bt1u+oC9G2n2Kd0HDhzAkiVLsHr1akRHR6OgoACzZ8/G4sWLMW/evGbbb1JSEnQ6neN97Tx3IiJqPLPZ7LglwMvLq7W7QxLk6ekJhUKBX375BWazGWq1usltuRVqAgMDIZfL6806Ki4uRnBwsMtt5s2bh2effRbPP/88ACAyMhJGoxHTpk3DG2+80ag2g4ODYTabUVZW5jRac6P9qlQqqFQqdw6PiIgacCv/eya6mdv1+XKrFaVSiaioKKebfu12O7KyshATE+Nym8rKynqdlcvlAKqHnRrTZlRUFBQKhVOd/Px8FBYWNrhfIiIiure4HY10Oh3WrVuHTZs2IS8vD9OnT4fRaMTkyZMBABMnTnS66XfkyJFYs2YN0tPTcfr0aezZswfz5s3DyJEjHeHmZm36+/tjypQp0Ol02L9/P3JzczF58mTExMQ0auYTERHRrQoPD0dqamqj6x84cACCIKCsrKzZ+gQAGzdurHfP6b3K7Xtqxo0bh0uXLmH+/PnQ6/Xo06cPMjMzHTf6FhYWOo3MvPnmmxAEAW+++SbOnz+P9u3bY+TIkfjLX/7S6DYB4J133oFMJsOYMWNgMpkQFxeH1atX38qxExGRBN3sRubk5GQsWLDA7XYPHz4Mb2/vRtcfOHAgioqK4O/v7/a+qGncfk7N3YrPqSEicl9VVRVOnz6NTp063dINnC2p7qzYLVu2YP78+cjPz3eU+fj4wMfHB0D1bRA2mw0eHnfvTyFu3LgRL7/8crOPCDWnG33O3Pn+5p1fREQkKcHBwY7F398fgiA43h8/fhy+vr7417/+haioKKhUKhw6dAgnT57E448/Do1GAx8fH/Tv3x979+51avf6y0+CIODDDz/E6NGj4eXlhW7dumH79u2O9ddffqq9TLRr1y5ERETAx8cHw4cPR1FRkWMbq9WKl156CQEBAWjXrh3mzJmD5557DqNGjXLrHKxZswZdunSBUqlEjx498NFHHznWiaKIBQsWoEOHDlCpVAgNDcVLL73kWL969Wp069YNarUaGo3G8dDbuwFDDRERNZooiqg0W1tluZ0XFubOnYulS5ciLy8PvXr1QkVFBUaMGIGsrCz85z//wfDhwzFy5EgUFhbesJ2FCxdi7Nix+P777zFixAjEx8ejtLS0wfqVlZVYvnw5PvroI3z11VcoLCzEq6++6lj/9ttvY/PmzdiwYQO+/vprGAwGfPHFF24d2+eff47Zs2fjlVdewbFjx/DCCy9g8uTJ2L9/PwDgf//3f/HOO+/g/fffx88//4wvvvgCkZGRAIB///vfeOmll7Bo0SLk5+cjMzMTDz30kFv7b01373gbERG1uKsWGx6Yv6tV9v3Tojh4KW/P19aiRYswbNgwx/u2bduid+/ejveLFy/G559/ju3bt2PmzJkNtjNp0iQ8/fTTAIAlS5bg3XffRU5ODoYPH+6yvsViwdq1a9GlSxcA1T8ltGjRIsf69957D0lJSRg9ejQAYOXKlcjIyHDr2JYvX45JkyZhxowZAKon43z77bdYvnw5hg4disLCQgQHByM2NhYKhQIdOnTAgAEDAFTfF+vt7Y0//vGP8PX1RceOHdG3b1+39t+aOFJDRET3nH79+jm9r6iowKuvvoqIiAgEBATAx8cHeXl5Nx2p6dWrl+O1t7c3/Pz8GvyBZ6D65wBqAw0AhISEOOqXl5ejuLjYETCA6kegREVFuXVseXl5GDRokFPZoEGDkJeXBwB46qmncPXqVXTu3BlTp07F559/DqvVCgAYNmwYOnbsiM6dO+PZZ5/F5s2b76qfx+BIDRERNZqnQo6fFsW12r5vl+tnMb366qvYs2cPli9fjq5du8LT0xNPPvkkzGbzDdtRKBRO7wVBgN1ud6t+S8/X0Wq1yM/Px969e7Fnzx7MmDEDy5Ytw8GDB+Hr64sjR47gwIED2L17N+bPn48FCxbg8OHDd8W0cY7UEBFRowmCAC+lR6sszfmbU19//TUmTZqE0aNHIzIyEsHBwThz5kyz7c8Vf39/aDQaHD582FFms9lw5MgRt9qJiIjA119/7VT29ddf44EHHnC89/T0xMiRI/Huu+/iwIEDyM7Oxg8//AAA8PDwQGxsLP7617/i+++/x5kzZ7Bv375bOLKWw5EaIiK653Xr1g3btm3DyJEjIQgC5s2bd8MRl+Yya9YspKSkoGvXrrj//vvx3nvv4ddff3Ur0L322msYO3Ys+vbti9jYWOzYsQPbtm1zzObauHEjbDYboqOj4eXlhY8//hienp7o2LEjvvzyS5w6dQoPPfQQ2rRpg4yMDNjtdvTo0aO5Dvm2YqghIqJ73ooVK/DnP/8ZAwcORGBgIObMmQODwdDi/ZgzZw70ej0mTpwIuVyOadOmIS4uzvEE/sYYNWoU/v73v2P58uWYPXs2OnXqhA0bNmDIkCEAgICAACxduhQ6nQ42mw2RkZHYsWMH2rVrh4CAAGzbtg0LFixAVVUVunXrhk8//RQPPvhgMx3x7cWH7xERUYPuxofvSYndbkdERATGjh2LxYsXt3Z3ms3tevgeR2qIiIjuEL/88gt2796Nhx9+GCaTCStXrsTp06cxYcKE1u7aXYE3ChMREd0hZDIZNm7ciP79+2PQoEH44YcfsHfvXkRERLR21+4KHKkhIiK6Q2i12nozl6jxOFJDREREksBQQ0RERJLAUENERESSwFBDREREksBQQ0RERJLAUENERESSwFBDRETUBGfOnIEgCDh69Giz72vjxo0t8ivZ4eHhSE1Nbfb9NBeGGiIikpxJkyZBEIR6y/Dhw1u7azflKliMGzcOJ06caJ0O3UX48D0iIpKk4cOHY8OGDU5lKpWqlXpzazw9PeHp6dna3bjjcaSGiIgkSaVSITg42Glp06YNAGDChAkYN26cU32LxYLAwED84x//AABkZmZi8ODBCAgIQLt27fDHP/4RJ0+ebHB/ri4RffHFFxAEwfH+5MmTePzxx6HRaODj44P+/ftj7969jvVDhgzBL7/8gsTERMfoUkNtr1mzBl26dIFSqUSPHj3w0UcfOa0XBAEffvghRo8eDS8vL3Tr1g3bt29v3MmrUVhYiMcffxw+Pj7w8/PD2LFjUVxc7Fj/3//+F0OHDoWvry/8/PwQFRWFf//73wCqf8dq5MiRaNOmDby9vfHggw8iIyPDrf27i6GGiIgaTxQBs7F1FlG8bYcRHx+PHTt2oKKiwlG2a9cuVFZWYvTo0QAAo9EInU6Hf//738jKyoJMJsPo0aNht9ubvN+KigqMGDECWVlZ+M9//oPhw4dj5MiRKCwsBABs27YN9913HxYtWoSioiIUFRW5bOfzzz/H7Nmz8corr+DYsWN44YUXMHnyZOzfv9+p3sKFCzF27Fh8//33GDFiBOLj41FaWtqovtrtdjz++OMoLS3FwYMHsWfPHpw6dcopDMbHx+O+++7D4cOHkZubi7lz50KhUAAAEhISYDKZ8NVXX+GHH37A22+/DR8fn6actkbj5SciImo8SyWwJLR19v36BUDp3ejqX375Zb0v0ddffx2vv/464uLi4O3tjc8//xzPPvssAOCTTz7Bn/70J/j6+gIAxowZ47Tt+vXr0b59e/z000/o2bNnkw6hd+/e6N27t+P94sWL8fnnn2P79u2YOXMm2rZtC7lcDl9fXwQHBzfYzvLlyzFp0iTMmDEDAKDT6fDtt99i+fLlGDp0qKPepEmT8PTTTwMAlixZgnfffRc5OTmNurcoKysLP/zwA06fPg2tVgsA+Mc//oEHH3wQhw8fRv/+/VFYWIjXXnsN999/PwCgW7duju0LCwsxZswYREZGAgA6d+7c2NPUZBypISIiSRo6dCiOHj3qtLz44osAAA8PD4wdOxabN28GUD0q889//hPx8fGO7X/++Wc8/fTT6Ny5M/z8/BAeHg4AjlGVpqioqMCrr76KiIgIBAQEwMfHB3l5eW63mZeXh0GDBjmVDRo0CHl5eU5lvXr1crz29vaGn58fLl682Oh9aLVaR6ABgAceeAABAQGO/eh0Ojz//POIjY3F0qVLnS7PvfTSS3jrrbcwaNAgJCcn4/vvv3frGJuCIzVERNR4Cq/qEZPW2rcbvL290bVr1wbXx8fH4+GHH8bFixexZ88eeHp6Oo1gjBw5Eh07dsS6desQGhoKu92Onj17wmw2u2xPJpNBvO4SmcVicXr/6quvYs+ePVi+fDm6du0KT09PPPnkkw22eatqLwXVEgThli6fXW/BggWYMGECdu7ciX/9619ITk5Geno6Ro8ejeeffx5xcXHYuXMndu/ejZSUFPztb3/DrFmzbtv+r8eRGiIiajxBqL4E1BpLnRtub4eBAwdCq9Viy5Yt2Lx5M5566ilHCLh8+TLy8/Px5ptv4pFHHkFERAR+/fXXG7bXvn17XLlyBUaj0VF2/TNsvv76a0yaNAmjR49GZGQkgoODcebMGac6SqUSNpvthvuKiIjA119/Xa/tBx544CZH3XgRERE4e/Yszp496yj76aefUFZW5rSf7t27IzExEbt378YTTzzhNONMq9XixRdfxLZt2/DKK69g3bp1t61/rnCkhoiIJMlkMkGv1zuVeXh4IDAw0PF+woQJWLt2LU6cOOF0k22bNm3Qrl07fPDBBwgJCUFhYSHmzp17w/1FR0fDy8sLr7/+Ol566SV899132Lhxo1Odbt26Ydu2bRg5ciQEQcC8efPqjZyEh4fjq6++wvjx46FSqZz6W+u1117D2LFj0bdvX8TGxmLHjh3Ytm2b00yqWxUbG4vIyEjEx8cjNTUVVqsVM2bMwMMPP4x+/frh6tWreO211/Dkk0+iU6dOOHfuHA4fPuy4F+nll1/Go48+iu7du+PXX3/F/v37ERERcdv65wpHaoiISJIyMzMREhLitAwePNipTnx8PH766SeEhYU53aMik8mQnp6O3Nxc9OzZE4mJiVi2bNkN99e2bVt8/PHHyMjIQGRkJD799FMsWLDAqc6KFSvQpk0bDBw4ECNHjkRcXBx+85vfONVZtGgRzpw5gy5duqB9+/Yu9zVq1Cj8/e9/x/Lly/Hggw/i/fffx4YNGzBkyJDGn6CbEAQB//znP9GmTRs89NBDiI2NRefOnbFlyxYAgFwux+XLlzFx4kR0794dY8eOxaOPPoqFCxcCAGw2GxISEhAREYHhw4eje/fuWL169W3rn8s+i9dfAJQog8EAf39/lJeXw8/Pr7W7Q0R0V6iqqsLp06fRqVMnqNXq1u4OSdSNPmfufH9zpIaIiIgkoUmhZtWqVQgPD4darUZ0dDRycnIarDtkyBCXv7/x2GOPOeq4Wi8IgtNQX3h4eL31S5cubUr3iYiISILcvlF4y5Yt0Ol0WLt2LaKjo5Gamoq4uDjk5+cjKCioXv1t27Y5TVW7fPkyevfujaeeespRdv0TE//1r39hypQp9R58tGjRIkydOtXxvvYBSURERERuh5oVK1Zg6tSpmDx5MgBg7dq12LlzJ9avX+/yzvC2bds6vU9PT4eXl5dTqLn+qYn//Oc/MXTo0HpPH7zZExaJiIjo3uXW5Sez2Yzc3FzExsZea0AmQ2xsLLKzsxvVRlpaGsaPHw9vb9ePui4uLsbOnTsxZcqUeuuWLl2Kdu3aoW/fvli2bBmsVmuD+zGZTDAYDE4LERERSZdbIzUlJSWw2WzQaDRO5RqNBsePH7/p9jk5OTh27BjS0tIarLNp0yb4+vriiSeecCp/6aWX8Jvf/AZt27bFN998g6SkJBQVFWHFihUu20lJSXFMKyMioltzj0yUpVZyuz5fLfrwvbS0NERGRmLAgAEN1lm/fj3i4+PrTenS6XSO17169YJSqcQLL7yAlJQUqFSqeu0kJSU5bWMwGJx+v4KIiG6u9gm7lZWV8PT0bOXekFRVVlYCqP+zDu5yK9QEBgZCLpejuLjYqby4uPim97oYjUakp6dj0aJFDdb5v//7P+Tn5zse7HMj0dHRsFqtOHPmDHr06FFvvUqlchl2iIio8eRyOQICAhw/gujl5QXhNv9cAd27RFFEZWUlLl68iICAAMjl8ltqz61Qo1QqERUVhaysLIwaNQoAYLfbkZWVhZkzZ95w261bt8JkMuGZZ55psE5aWhqioqKcfpa9IUePHoVMJnM544qIiG6f2v+0NvbXnYncFRAQcFsmArl9+Umn0+G5555Dv379MGDAAKSmpsJoNDpmQ02cOBFhYWFISUlx2i4tLQ2jRo1Cu3btXLZrMBiwdetW/O1vf6u3Ljs7G9999x2GDh0KX19fZGdnIzExEc888wzatGnj7iEQEZEbBEFASEgIgoKC6v3qNNGtUigUtzxCU8vtUDNu3DhcunQJ8+fPh16vR58+fZCZmem4ebiwsBAymfOkqvz8fBw6dAi7d+9usN309HSIooinn3663jqVSoX09HQsWLAAJpMJnTp1QmJiotM9M0RE1Lzkcvlt+/Ihag787SciIiK6Y/G3n4iIiOiew1BDREREksBQQ0RERJLAUENERESSwFBDREREksBQQ0RERJLAUENERESSwFBDREREksBQQ0RERJLAUENERESSwFBDREREksBQQ0RERJLAUENERESSwFBDREREksBQQ0RERJLAUENERESSwFBDREREksBQQ0RERJLAUENERESSwFBDREREksBQQ0RERJLAUENERESSwFBDREREksBQQ0RERJLAUENERESSwFBDREREksBQQ0RERJLAUENERESSwFBDREREksBQQ0RERJLAUENERESS0KRQs2rVKoSHh0OtViM6Oho5OTkN1h0yZAgEQai3PPbYY446kyZNqrd++PDhTu2UlpYiPj4efn5+CAgIwJQpU1BRUdGU7hMREZEEuR1qtmzZAp1Oh+TkZBw5cgS9e/dGXFwcLl686LL+tm3bUFRU5FiOHTsGuVyOp556yqne8OHDnep9+umnTuvj4+Px448/Ys+ePfjyyy/x1VdfYdq0ae52n4iIiCRKEEVRdGeD6Oho9O/fHytXrgQA2O12aLVazJo1C3Pnzr3p9qmpqZg/fz6Kiorg7e0NoHqkpqysDF988YXLbfLy8vDAAw/g8OHD6NevHwAgMzMTI0aMwLlz5xAaGnrT/RoMBvj7+6O8vBx+fn6NPFoiIiJqTe58f7s1UmM2m5Gbm4vY2NhrDchkiI2NRXZ2dqPaSEtLw/jx4x2BptaBAwcQFBSEHj16YPr06bh8+bJjXXZ2NgICAhyBBgBiY2Mhk8nw3XffudyPyWSCwWBwWoiIiEi63Ao1JSUlsNls0Gg0TuUajQZ6vf6m2+fk5ODYsWN4/vnnncqHDx+Of/zjH8jKysLbb7+NgwcP4tFHH4XNZgMA6PV6BAUFOW3j4eGBtm3bNrjflJQU+Pv7OxatVuvOoRIREdFdxqMld5aWlobIyEgMGDDAqXz8+PGO15GRkejVqxe6dOmCAwcO4JFHHmnSvpKSkqDT6RzvDQYDgw0REZGEuTVSExgYCLlcjuLiYqfy4uJiBAcH33Bbo9GI9PR0TJky5ab76dy5MwIDA1FQUAAACA4OrncjstVqRWlpaYP7ValU8PPzc1qIiIhIutwKNUqlElFRUcjKynKU2e12ZGVlISYm5obbbt26FSaTCc8888xN93Pu3DlcvnwZISEhAICYmBiUlZUhNzfXUWffvn2w2+2Ijo525xCIiIhIotye0q3T6bBu3Tps2rQJeXl5mD59OoxGIyZPngwAmDhxIpKSkuptl5aWhlGjRqFdu3ZO5RUVFXjttdfw7bff4syZM8jKysLjjz+Orl27Ii4uDgAQERGB4cOHY+rUqcjJycHXX3+NmTNnYvz48Y2a+URERETS5/Y9NePGjcOlS5cwf/586PV69OnTB5mZmY6bhwsLCyGTOWel/Px8HDp0CLt3767Xnlwux/fff49NmzahrKwMoaGh+MMf/oDFixdDpVI56m3evBkzZ87EI488AplMhjFjxuDdd991t/tEREQkUW4/p+ZuxefUEBER3X2a7Tk1RERERHcqhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpKEJoWaVatWITw8HGq1GtHR0cjJyWmw7pAhQyAIQr3lscceAwBYLBbMmTMHkZGR8Pb2RmhoKCZOnIgLFy44tRMeHl6vjaVLlzal+0RERCRBboeaLVu2QKfTITk5GUeOHEHv3r0RFxeHixcvuqy/bds2FBUVOZZjx45BLpfjqaeeAgBUVlbiyJEjmDdvHo4cOYJt27YhPz8ff/rTn+q1tWjRIqe2Zs2a5W73iYiISKI83N1gxYoVmDp1KiZPngwAWLt2LXbu3In169dj7ty59eq3bdvW6X16ejq8vLwcocbf3x979uxxqrNy5UoMGDAAhYWF6NChg6Pc19cXwcHB7naZiIiI7gFujdSYzWbk5uYiNjb2WgMyGWJjY5Gdnd2oNtLS0jB+/Hh4e3s3WKe8vByCICAgIMCpfOnSpWjXrh369u2LZcuWwWq1NtiGyWSCwWBwWoiIiEi63BqpKSkpgc1mg0ajcSrXaDQ4fvz4TbfPycnBsWPHkJaW1mCdqqoqzJkzB08//TT8/Pwc5S+99BJ+85vfoG3btvjmm2+QlJSEoqIirFixwmU7KSkpWLhwYSOPjIiIiO52bl9+uhVpaWmIjIzEgAEDXK63WCwYO3YsRFHEmjVrnNbpdDrH6169ekGpVOKFF15ASkoKVCpVvbaSkpKctjEYDNBqtbfpSIiIiOhO49blp8DAQMjlchQXFzuVFxcX3/ReF6PRiPT0dEyZMsXl+tpA88svv2DPnj1OozSuREdHw2q14syZMy7Xq1Qq+Pn5OS1EREQkXW6FGqVSiaioKGRlZTnK7HY7srKyEBMTc8Ntt27dCpPJhGeeeabeutpA8/PPP2Pv3r1o167dTfty9OhRyGQyBAUFuXMIREREJFFuX37S6XR47rnn0K9fPwwYMACpqakwGo2O2VATJ05EWFgYUlJSnLZLS0vDqFGj6gUWi8WCJ598EkeOHMGXX34Jm80GvV4PoHrmlFKpRHZ2Nr777jsMHToUvr6+yM7ORmJiIp555hm0adOmqcdOREREEuJ2qBk3bhwuXbqE+fPnQ6/Xo0+fPsjMzHTcPFxYWAiZzHkAKD8/H4cOHcLu3bvrtXf+/Hls374dANCnTx+ndfv378eQIUOgUqmQnp6OBQsWwGQyoVOnTkhMTHS6Z4aIiIjubYIoimJrd6IlGAwG+Pv7o7y8nPfXEBER3SXc+f7mbz8RERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJHi0dgfudieKr+DFj3LRKdAbndt7o1OgDzoFeqNLe2+091VBEITW7iIREdE9gaHmFhVcrMCpEiNOlRiRddx5nY/KA50CvesEHm90ae+D8EBv+Kh46omIiG4nQRRFsbU70RIMBgP8/f1RXl4OPz+/29ZueaUFP14orw42l4w4XVIdcs6WVsJ+gzMb5KtyjOx0aV8bfHxwXxtPKOS8KkhERAS49/3NUNNMzFY7Ckurg86pEiNOXzLidIkRp0oqUFJhbnA7D5mADm29HCM7ndv7OEZ62vvwchYREd1b3Pn+5jWQZqL0kKFrkC+6BvnWW1d+1YLTJTWjOteFnqsWm+Ny1vVqL2fVDTyday5vefNyFhER3eM4UnMHsdtFFF+pcgSdU5cqasLPzS9nafxUTkGn9tKWto0nPHg5i4iI7lLNfvlp1apVWLZsGfR6PXr37o333nsPAwYMcFl3yJAhOHjwYL3yESNGYOfOnQAAURSRnJyMdevWoaysDIMGDcKaNWvQrVs3R/3S0lLMmjULO3bsgEwmw5gxY/D3v/8dPj4+jerz3RBqbsRkteFsaSVO1l7GqhN4bno5q51XTdCpuZQV6I1OvJxFRER3gWYNNVu2bMHEiROxdu1aREdHIzU1FVu3bkV+fj6CgoLq1S8tLYXZfO1L9/Lly+jduzc+/PBDTJo0CQDw9ttvIyUlBZs2bUKnTp0wb948/PDDD/jpp5+gVqsBAI8++iiKiorw/vvvw2KxYPLkyejfvz8++eSTRvX7bg81N1JeacHpy9eCjuOSVkkFqiz2BrfzVXmgU+2lrEAfdGrvzctZRER0R2nWUBMdHY3+/ftj5cqVAAC73Q6tVotZs2Zh7ty5N90+NTUV8+fPR1FREby9vSGKIkJDQ/HKK6/g1VdfBQCUl5dDo9Fg48aNGD9+PPLy8vDAAw/g8OHD6NevHwAgMzMTI0aMwLlz5xAaGnrT/Uo51DTEbhehN1Q5RnauzdAy4tyvN76cFeynvu7+nergcx8vZxERUQtqthuFzWYzcnNzkZSU5CiTyWSIjY1FdnZ2o9pIS0vD+PHj4e3tDQA4ffo09Ho9YmNjHXX8/f0RHR2N7OxsjB8/HtnZ2QgICHAEGgCIjY2FTCbDd999h9GjR9fbj8lkgslkcrw3GAzuHKokyGQCQgM8ERrgiUFdA53Wmaw2FF6udJ6KXhN4LhvN0BuqoDdUIfvUZaftFPLq2Vl1p6LX3ssT6KPk5SwiImo1boWakpIS2Gw2aDQap3KNRoPjx483sNU1OTk5OHbsGNLS0hxler3e0cb1bdau0+v19S5teXh4oG3bto4610tJScHChQtvflD3KJWHHN00vuimcTE7q9KCUyXXLmWdLjHi5KUKnLlsRJXFjpOXjDh5yYi9ec7b+ao86k1Frx3l8VI28+Usux2wXgUsVwFLJWCpqvnzap3yOsv1dW0mQOkDqAMAtX/Di9IbYHAjIrojteiNE2lpaYiMjGzwpuLbKSkpCTqdzvHeYDBAq9U2+36lwN9Lgb4d2qBvhzZO5Xa7iCJDVc3084prNy2XVODcr1dxxWTFf8+V47/nymu2EKGEFWqY0cFXQLe2MnTylyPcXwatDxDqDQSqbJDbqlwEkprX1ioXIaXO69rFZqp/IM1BkN849LgKRZ51yhReDEVERM3ErVATGBgIuVyO4uJip/Li4mIEBwffcFuj0Yj09HQsWrTIqbx2u+LiYoSEhDi12adPH0edixcvOm1ntVpRWlra4H5VKhVUKlWjjuueY7c1HA4cQaL+iIfMehVhlqsIs1RicG3w8KgEAqtg96uExVQJu6kSoqUSMlsVFHYT5Ki5UdkCoLhmaW5yFaDwrLN4AR7qa68V6po/PQGPmjpyBWCuAKrKG17sVkC0AVdLq5emkHncJBQ1EIwYioiIbsqtUKNUKhEVFYWsrCyMGjUKQPWNwllZWZg5c+YNt926dStMJhOeeeYZp/JOnTohODgYWVlZjhBjMBjw3XffYfr06QCAmJgYlJWVITc3F1FRUQCAffv2wW63Izo62p1DuHOJImAzuwgZri6fVF4XSG5yqeX6+raGp4A3lQzAjSKkKMhglalhlqlwVVTBaFfgik0Bo6iESVTgKlSoghJXRSWuQgUTlLB5qOHt5QtfX1/4+/mhbYA/AtsEIKhdANSevjVBxcs5wHioAZn8th8fRLH6/DUYesquvb5a5rqOaKsORpWXq5emcBmKAhofjBSeDEVEJFluX37S6XR47rnn0K9fPwwYMACpqakwGo2YPHkyAGDixIkICwtDSkqK03ZpaWkYNWoU2rVr51QuCAJefvllvPXWW+jWrZtjSndoaKgjOEVERGD48OGYOnUq1q5dC4vFgpkzZ2L8+PGNmvnUrAxFwJlDtyd4oBWeg+ihdh0O6o5i1BvdcBUmrq9bd4TEC4JcAYUgQAHAG0Agrl3OOnWpAsXXTUU/9+tViFYAVQDqDYqYEeJvRKdAoHN7OToFKtC5vR2dA4GwAKF5rqkKQvX9NEpvwK8JnzlRBMzGG48E1Q1GzRaKFI0YKXIRimovoXmoGYqI6I7l9r//48aNw6VLlzB//nzo9Xr06dMHmZmZjht9CwsLIZM5T/nNz8/HoUOHsHv3bpdt/r//9/9gNBoxbdo0lJWVYfDgwcjMzHQ8owYANm/ejJkzZ+KRRx5xPHzv3Xffdbf7t1/xMWDb87e3TUFeJzS4uFTS1OBRJ2RcG9VovenZMpmAsABPhAV44nfd2jutq7LYUFhaWW8q+ukSI0qNZhSVV6GovArfnHT+clfKZejQzqvONPRrNy23827F2VmCAKh8qhf/MPe3v2koKrt5MBLtgN0CVJZUL00hVzYtFNUuDEVE1Iz4Mwm3qui/wO43bxw8rg8T9QLJdXXlitvXPwkqqzS7nIp+usQIk7Xhhw36qT3QqfZnJGqeqtw50AfhgV7NPzurtYnize8ZakwoulUNhqKAxpUp1DfbAxFJDH+l24V78eF79xq7XcSF8qv1pqKfLjHifNlV3OiTHuSrgtJDBkEABAiQCdWXRgUAEAAB194LAiCrGW2oW1Z3W9Qtr1NPVlNwbZ3gtK0gXNem0z6EOmXXbVu3TpO3rVPutI0AASJU9kqobRXwtFdAbauA2maAp80ItfUK1LYrNWUVUFmvQGWrgNp6BSprBVS2K1BaKyDDrYcim0wJs4cvLApfmBV+sCh8Yal5b1X4QFCoIXioIVOoIFOoIVN6Qq5QQ65Uw0PpCQ+lGh4qTyiUnlCoPCFXegIequqbyz1U1f/5kCs4mkR0B2GocYGh5t5WZbHhl8uVzlPRawLPr5WW1u6e5AmwwxtV8EMl/IRK+MF43Z+V8BOMN1wvE1rmnyo7BFiggEVQwiIoYRWUsMkUsMqUsMlUsMmUsMuVEGUq2OUqiNeHoppQJShUkHl4QqasDlgeyuoQ5QhWKjUUKi8olNVBrHrbmnbkSgYrohrN9kRhoruVWiFHj2Bf9Aiu/7DBX41mnPv1Kqx2O0RUX6kRRdH1a4g1ZXVeo26duusb0U7dbdDQtnXrNrCtU9su2qzX/0a06e62Nzs+V22KIsoBlN2k/7DboRQr4WmtHinytF2p/tNuhJftCjztRqhsRsjsZsjtJsjsFnjYTfCwm+EhmuEhWqAQzVCKZihhgVKwQgUzVLBUL4LV8XmQQaxeJ5qv3btvu52fxsYxQQmLoIC1TriyypSwy5SwyZSwyWtCVU3AEh2hSlkdkmpGreQKFQSFGvKaUSsP5bWApVB5QqFWQ6H0hIfS07F99YgVgxXdfRhq6J7XxluJNt7K1u4GtQBRFGG1izBZ7TBb7TBabSi12GG2WmGuqoLVfBUW01VYzFWwma/Car4Ku7kKNksV7JYqiBYT7JarEK0mwGoCrFUQrCbAZoJgNUFmN0Nmq4LMboHcboLcVhOqasKVQjRDAQsUogUqwVITrKw1wcp5xPBasDK20tm6FqyqQ5UCFkEFm0xRHapkKthlStjrhiunESsVBA8VBA81BIUaMqUacg81ZCpPeCjUkKs84aFUwUPpCYXKC0qVJ5S+gRC82jJMUZMx1BDRPUMQBCjkAhRymYsHK/m3WD/sdhFmmx0ma/VisNpgtlhhNlXBbK6C1XQVVtPV6mBlMcFmrnSEK9Figt1aBdFcBdFWBVjMgK06XAk2EwSbGTKbCTK7CXKbCXLRUh2q7CbnUStYoKwdqXKMWLV+sDKKKhQJ7aEXgnBJHoTLcg0uK4JRrtTAoA6BSdkOSqUCag851AoZVDV/qhX136s85FApZI661WW1da/Vl8sYoqSCoYaIqIXJZALUsuovVmcte7+f1WaH2VY9alVmtcNktsFsuQpzVfUolaVm9MpmvgqbuXq06tqoVfVit5ogWKuAmlAFqxmCrQoyuwkym9lxSdCjZtRKbq8draq+HKhwhCoz/ISr8BZM6Ipz6CqeA6yoXur8CopJVOC82A7nxUCcE9vjvBjo9FqPtrDDvUdVKOQC1B5yqByhp+EAdO3Puuuvry+rae9a3esDl0Leeo/TkDKGGiKie5SHXAYPuQxeTldfvVu0D6IowmITYbLaUFJVCWtpIexlZ4Fff4FQfg7yK+egvHIOKuN5qK8WQyVY0FnQozNc/5ixDTKUegThkiwIxbIgFKE9LiAQhfZA/GILRKG1DYxWGcy2a7PxLDYRFpsVV0xWl202B7lMuBaCav5UNhSSasNUTV2VwkWoqh2Vqlu/7nYKGZRyWes9q6uFcPYTERHdHWwWwHAeKDsLlJ8FygprXhdWvy4/X/2AyRsSAN8QiP5a2Pzug8X3Pph9wnDVKwxGz1BcUQejCipUWWyosthhstpgsthRZbU5lVVZ7PXem2rqmKzX1tV9f6PnaLUEQYBTUKp7aU5VNxTVHbFyMSqlun4Uy+NacPL3VCDE3/O29puzn4iISHrkCqBNePXiit0GVBTXDzt1Q5C1CrhyAcKVC/DAd/AA4Inr7qjyCgQCOgABWsBfCwR0BNrWvu4AqJv2H2NRrL5JvW5IaigAVVlsqLLaYbqurMH6LupW1eynduhCFFGznR3VvzJ8+w3uGoiPn2+932RkqCEiImmQyat/m80vFOjw2/rrRREwlrgOO2U1f5qvXPspkQtHXO9H7V8dbvxrgk9Ah5rAUxOAPNu4nMElCIJjdMMfLfPk+NrLe44Qdd1IU8Mh6bqRKlehqs4oVm0df6/WfSI+Lz8REREBNUMZZdcCTvnZmte/XHt9td4v7Nan8HYRduqEIO+gVv3dvbsNLz8RERG5SxCqR1k82wAhvVzXMVW4Dju1IaiiGLAYgUvHqxdX5CrA/776Yac2BPmGAHJ+PTcFzxoREVFjqXyAoIjqxRVLFVB+ruYS1/UjPoXAlQuAzQSUnqxeXBHkgH+Yi8tbNe/97qt+8jPVw1BDRER0uyjUQGDX6sUVmwUwXKgfdmpDUPm56hlcZTX3/fziqpHqGVzXbmSuvam5Q00Aug9QejXnUd6xGGqIiIhailwBtOlYvbjimMF1XdhxhKBrM7hw5QJw9jvX7biawRVw6zO47nQMNURERHcKpxlcLqZG384ZXLUjO9eP+NxgBtedjqGGiIjobiEIgE/76iUsqv76ujO4XD2gsHYGV1U5UPUDUPyD6/3cpTO4GGqIiIikwt0ZXK5GfG5lBldgN9dhq4Uw1BAREd1LGjODy3C+esp6vRGfs9XrGprB1XEwMHln8x9DAxhqiIiI6BqFGmjXpXpxpXYGl6vLW6F9W7av12GoISIiosa72QyuVnTn3eVDRERE1AQMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAlNCjWrVq1CeHg41Go1oqOjkZOTc8P6ZWVlSEhIQEhICFQqFbp3746MjAzH+vDwcAiCUG9JSEhw1BkyZEi99S+++GJTuk9EREQS5PZvP23ZsgU6nQ5r165FdHQ0UlNTERcXh/z8fAQFBdWrbzabMWzYMAQFBeGzzz5DWFgYfvnlFwQEBDjqHD58GDabzfH+2LFjGDZsGJ566imntqZOnYpFixY53nt5ebnbfSIiIpIot0PNihUrMHXqVEyePBkAsHbtWuzcuRPr16/H3Llz69Vfv349SktL8c0330ChUACoHpmpq3379k7vly5dii5duuDhhx92Kvfy8kJwcLC7XSYiIqJ7gFuXn8xmM3JzcxEbG3utAZkMsbGxyM7OdrnN9u3bERMTg4SEBGg0GvTs2RNLlixxGpm5fh8ff/wx/vznP0MQBKd1mzdvRmBgIHr27ImkpCRUVlY22FeTyQSDweC0EBERkXS5NVJTUlICm80GjUbjVK7RaHD8+HGX25w6dQr79u1DfHw8MjIyUFBQgBkzZsBisSA5Oble/S+++AJlZWWYNGmSU/mECRPQsWNHhIaG4vvvv8ecOXOQn5+Pbdu2udxvSkoKFi5c6M7hERER0V3M7ctP7rLb7QgKCsIHH3wAuVyOqKgonD9/HsuWLXMZatLS0vDoo48iNDTUqXzatGmO15GRkQgJCcEjjzyCkydPokuXLvXaSUpKgk6nc7w3GAzQarW38ciIiIjoTuJWqAkMDIRcLkdxcbFTeXFxcYP3uoSEhEChUEAulzvKIiIioNfrYTaboVQqHeW//PIL9u7d2+DoS13R0dEAgIKCApehRqVSQaVSNeq4iIiI6O7n1j01SqUSUVFRyMrKcpTZ7XZkZWUhJibG5TaDBg1CQUEB7Ha7o+zEiRMICQlxCjQAsGHDBgQFBeGxxx67aV+OHj0KoDo0EREREbn9nBqdTod169Zh06ZNyMvLw/Tp02E0Gh2zoSZOnIikpCRH/enTp6O0tBSzZ8/GiRMnsHPnTixZssTpGTRAdTjasGEDnnvuOXh4OA8gnTx5EosXL0Zubi7OnDmD7du3Y+LEiXjooYfQq1evphw3ERERSYzb99SMGzcOly5dwvz586HX69GnTx9kZmY6bh4uLCyETHYtK2m1WuzatQuJiYno1asXwsLCMHv2bMyZM8ep3b1796KwsBB//vOf6+1TqVRi7969SE1NhdFohFarxZgxY/Dmm2+6230iIiKSKEEURbG1O9ESDAYD/P39UV5eDj8/v9buDhERETWCO9/f/O0nIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikoQmhZpVq1YhPDwcarUa0dHRyMnJuWH9srIyJCQkICQkBCqVCt27d0dGRoZj/YIFCyAIgtNy//33O7VRVVWFhIQEtGvXDj4+PhgzZgyKi4ub0n0iIiKSILdDzZYtW6DT6ZCcnIwjR46gd+/eiIuLw8WLF13WN5vNGDZsGM6cOYPPPvsM+fn5WLduHcLCwpzqPfjggygqKnIshw4dclqfmJiIHTt2YOvWrTh48CAuXLiAJ554wt3uExERkUR5uLvBihUrMHXqVEyePBkAsHbtWuzcuRPr16/H3Llz69Vfv349SktL8c0330ChUAAAwsPD63fEwwPBwcEu91leXo60tDR88skn+P3vfw8A2LBhAyIiIvDtt9/it7/9rbuHQURERBLj1kiN2WxGbm4uYmNjrzUgkyE2NhbZ2dkut9m+fTtiYmKQkJAAjUaDnj17YsmSJbDZbE71fv75Z4SGhqJz586Ij49HYWGhY11ubi4sFovTfu+//3506NChwf2aTCYYDAanhYiIiKTLrVBTUlICm80GjUbjVK7RaKDX611uc+rUKXz22Wew2WzIyMjAvHnz8Le//Q1vvfWWo050dDQ2btyIzMxMrFmzBqdPn8bvfvc7XLlyBQCg1+uhVCoREBDQ6P2mpKTA39/fsWi1WncOlYiIiO4ybl9+cpfdbkdQUBA++OADyOVyREVF4fz581i2bBmSk5MBAI8++qijfq9evRAdHY2OHTvif/7nfzBlypQm7TcpKQk6nc7x3mAwMNgQERFJmFuhJjAwEHK5vN6so+Li4gbvhwkJCYFCoYBcLneURUREQK/Xw2w2Q6lU1tsmICAA3bt3R0FBAQAgODgYZrMZZWVlTqM1N9qvSqWCSqVy5/CIiIjoLubW5SelUomoqChkZWU5yux2O7KyshATE+Nym0GDBqGgoAB2u91RduLECYSEhLgMNABQUVGBkydPIiQkBAAQFRUFhULhtN/8/HwUFhY2uF8iIiK6t7g9pVun02HdunXYtGkT8vLyMH36dBiNRsdsqIkTJyIpKclRf/r06SgtLcXs2bNx4sQJ7Ny5E0uWLEFCQoKjzquvvoqDBw/izJkz+OabbzB69GjI5XI8/fTTAAB/f39MmTIFOp0O+/fvR25uLiZPnoyYmBjOfCIiIiIATbinZty4cbh06RLmz58PvV6PPn36IDMz03HzcGFhIWSya1lJq9Vi165dSExMRK9evRAWFobZs2djzpw5jjrnzp3D008/jcuXL6N9+/YYPHgwvv32W7Rv395R55133oFMJsOYMWNgMpkQFxeH1atX38qxExERkYQIoiiKrd2JlmAwGODv74/y8nL4+fm1dneIiIioEdz5/uZvPxEREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQw1BAREZEkMNQQERGRJDDUEBERkSQ0KdSsWrUK4eHhUKvViI6ORk5Ozg3rl5WVISEhASEhIVCpVOjevTsyMjIc61NSUtC/f3/4+voiKCgIo0aNQn5+vlMbQ4YMgSAITsuLL77YlO4TERGRBLkdarZs2QKdTofk5GQcOXIEvXv3RlxcHC5evOiyvtlsxrBhw3DmzBl89tlnyM/Px7p16xAWFuaoc/DgQSQkJODbb7/Fnj17YLFY8Ic//AFGo9GpralTp6KoqMix/PWvf3W3+0RERCRRgiiKojsbREdHo3///li5ciUAwG63Q6vVYtasWZg7d269+mvXrsWyZctw/PhxKBSKRu3j0qVLCAoKwsGDB/HQQw8BqB6p6dOnD1JTU93proPBYIC/vz/Ky8vh5+fXpDaIiIioZbnz/e3WSI3ZbEZubi5iY2OvNSCTITY2FtnZ2S632b59O2JiYpCQkACNRoOePXtiyZIlsNlsDe6nvLwcANC2bVun8s2bNyMwMBA9e/ZEUlISKisr3ek+ERERSZiHO5VLSkpgs9mg0WicyjUaDY4fP+5ym1OnTmHfvn2Ij49HRkYGCgoKMGPGDFgsFiQnJ9erb7fb8fLLL2PQoEHo2bOno3zChAno2LEjQkND8f3332POnDnIz8/Htm3bXO7XZDLBZDI53hsMBncOlYiIiO4yboWaprDb7QgKCsIHH3wAuVyOqKgonD9/HsuWLXMZahISEnDs2DEcOnTIqXzatGmO15GRkQgJCcEjjzyCkydPokuXLvXaSUlJwcKFC2//AREREdEdya3LT4GBgZDL5SguLnYqLy4uRnBwsMttQkJC0L17d8jlckdZREQE9Ho9zGazU92ZM2fiyy+/xP79+3HffffdsC/R0dEAgIKCApfrk5KSUF5e7ljOnj170+MjIiKiu5dboUapVCIqKgpZWVmOMrvdjqysLMTExLjcZtCgQSgoKIDdbneUnThxAiEhIVAqlQAAURQxc+ZMfP7559i3bx86dep0074cPXoUQHVockWlUsHPz89pISIiIulye0q3TqfDunXrsGnTJuTl5WH69OkwGo2YPHkyAGDixIlISkpy1J8+fTpKS0sxe/ZsnDhxAjt37sSSJUuQkJDgqJOQkICPP/4Yn3zyCXx9faHX66HX63H16lUAwMmTJ7F48WLk5ubizJkz2L59OyZOnIiHHnoIvXr1utVzQERERBLg9j0148aNw6VLlzB//nzo9Xr06dMHmZmZjpuHCwsLIZNdy0parRa7du1CYmIievXqhbCwMMyePRtz5sxx1FmzZg2A6mnbdW3YsAGTJk2CUqnE3r17kZqaCqPRCK1WizFjxuDNN99syjETERGRBLn9nJq7FZ9TQ0REdPdptufUEBEREd2pGGqIiIhIEhhqiIiISBIYaoiIiEgSGGqIiIhIEhhqiIiISBIYaoiIiEgSGGqIiIhIEhhqiIiISBIYaoiIiEgSGGqIiIhIEhhqiIiISBIYaoiIiEgSGGqIiIhIEhhqiIiISBIYaoiIiEgSGGqIiIhIEhhqiIiISBIYaoiIiEgSGGqIiIhIEhhqiIiISBIYaoiIiEgSGGqIiIhIEhhqiIiISBIYaoiIiEgSGGqIiIhIEhhqiIiISBIYaoiIiEgSGGqIiIhIEhhqiIiISBIYaoiIiEgSmhRqVq1ahfDwcKjVakRHRyMnJ+eG9cvKypCQkICQkBCoVCp0794dGRkZbrVZVVWFhIQEtGvXDj4+PhgzZgyKi4ub0n0iIiKSILdDzZYtW6DT6ZCcnIwjR46gd+/eiIuLw8WLF13WN5vNGDZsGM6cOYPPPvsM+fn5WLduHcLCwtxqMzExETt27MDWrVtx8OBBXLhwAU888UQTDpmIiIikSBBFUXRng+joaPTv3x8rV64EANjtdmi1WsyaNQtz586tV3/t2rVYtmwZjh8/DoVC0aQ2y8vL0b59e3zyySd48sknAQDHjx9HREQEsrOz8dvf/vam/TYYDPD390d5eTn8/PzcOWQiIiJqJe58f7s1UmM2m5Gbm4vY2NhrDchkiI2NRXZ2tstttm/fjpiYGCQkJECj0aBnz55YsmQJbDZbo9vMzc2FxWJxqnP//fejQ4cODe6XiIiI7i0e7lQuKSmBzWaDRqNxKtdoNDh+/LjLbU6dOoV9+/YhPj4eGRkZKCgowIwZM2CxWJCcnNyoNvV6PZRKJQICAurV0ev1LvdrMplgMpkc7w0GgzuHSkRERHeZZp/9ZLfbERQUhA8++ABRUVEYN24c3njjDaxdu7ZZ95uSkgJ/f3/HotVqm3V/RERE1LrcCjWBgYGQy+X1Zh0VFxcjODjY5TYhISHo3r075HK5oywiIgJ6vR5ms7lRbQYHB8NsNqOsrKzR+01KSkJ5ebljOXv2rDuHSkRERHcZt0KNUqlEVFQUsrKyHGV2ux1ZWVmIiYlxuc2gQYNQUFAAu93uKDtx4gRCQkKgVCob1WZUVBQUCoVTnfz8fBQWFja4X5VKBT8/P6eFiIiIpMvty086nQ7r1q3Dpk2bkJeXh+nTp8NoNGLy5MkAgIkTJyIpKclRf/r06SgtLcXs2bNx4sQJ7Ny5E0uWLEFCQkKj2/T398eUKVOg0+mwf/9+5ObmYvLkyYiJiWnUzCciIiKSPrduFAaAcePG4dKlS5g/fz70ej369OmDzMxMx42+hYWFkMmuZSWtVotdu3YhMTERvXr1QlhYGGbPno05c+Y0uk0AeOeddyCTyTBmzBiYTCbExcVh9erVt3LsREREJCFuP6fmbsXn1BAREd19mu05NURERER3KoYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEhhoiIiKSBIYaIiIikgSGGiIiIpIEj9buQEsRRREAYDAYWrknRERE1Fi139u13+M3cs+EmitXrgAAtFptK/eEiIiI3HXlyhX4+/vfsI4gNib6SIDdbseFCxfg6+sLQRBauzu3xGAwQKvV4uzZs/Dz82vt7twxeF4axnPjGs9Lw3huXON5aVhznRtRFHHlyhWEhoZCJrvxXTP3zEiNTCbDfffd19rduK38/Pz4l8oFnpeG8dy4xvPSMJ4b13heGtYc5+ZmIzS1eKMwERERSQJDDREREUkCQ81dSKVSITk5GSqVqrW7ckfheWkYz41rPC8N47lxjeelYXfCublnbhQmIiIiaeNIDREREUkCQw0RERFJAkMNERERSQJDDREREUkCQ80datWqVQgPD4darUZ0dDRycnIarLtu3Tr87ne/Q5s2bdCmTRvExsbesP7dzJ3zsm3bNvTr1w8BAQHw9vZGnz598NFHH7Vgb1uWO+emrvT0dAiCgFGjRjVvB1uJO+dl48aNEATBaVGr1S3Y25bl7memrKwMCQkJCAkJgUqlQvfu3ZGRkdFCvW057pyXIUOG1PvMCIKAxx57rAV73HLc/cykpqaiR48e8PT0hFarRWJiIqqqqpqvgyLdcdLT00WlUimuX79e/PHHH8WpU6eKAQEBYnFxscv6EyZMEFetWiX+5z//EfPy8sRJkyaJ/v7+4rlz51q4583L3fOyf/9+cdu2beJPP/0kFhQUiKmpqaJcLhczMzNbuOfNz91zU+v06dNiWFiY+Lvf/U58/PHHW6azLcjd87JhwwbRz89PLCoqcix6vb6Fe90y3D03JpNJ7NevnzhixAjx0KFD4unTp8UDBw6IR48ebeGeNy93z8vly5edPi/Hjh0T5XK5uGHDhpbteAtw99xs3rxZVKlU4ubNm8XTp0+Lu3btEkNCQsTExMRm6yNDzR1owIABYkJCguO9zWYTQ0NDxZSUlEZtb7VaRV9fX3HTpk3N1cVWcavnRRRFsW/fvuKbb77ZHN1rVU05N1arVRw4cKD44Ycfis8995wkQ42752XDhg2iv79/C/Wudbl7btasWSN27txZNJvNLdXFVnGr/8688847oq+vr1hRUdFcXWw17p6bhIQE8fe//71TmU6nEwcNGtRsfeTlpzuM2WxGbm4uYmNjHWUymQyxsbHIzs5uVBuVlZWwWCxo27Ztc3Wzxd3qeRFFEVlZWcjPz8dDDz3UnF1tcU09N4sWLUJQUBCmTJnSEt1scU09LxUVFejYsSO0Wi0ef/xx/Pjjjy3R3RbVlHOzfft2xMTEICEhARqNBj179sSSJUtgs9laqtvN7nb8+5uWlobx48fD29u7ubrZKppybgYOHIjc3FzHJapTp04hIyMDI0aMaLZ+3jM/aHm3KCkpgc1mg0ajcSrXaDQ4fvx4o9qYM2cOQkNDnT58d7umnpfy8nKEhYXBZDJBLpdj9erVGDZsWHN3t0U15dwcOnQIaWlpOHr0aAv0sHU05bz06NED69evR69evVBeXo7ly5dj4MCB+PHHHyX1g7hNOTenTp3Cvn37EB8fj4yMDBQUFGDGjBmwWCxITk5uiW43u1v99zcnJwfHjh1DWlpac3Wx1TTl3EyYMAElJSUYPHgwRFGE1WrFiy++iNdff73Z+slQIzFLly5Feno6Dhw4IOkbHBvL19cXR48eRUVFBbKysqDT6dC5c2cMGTKktbvWaq5cuYJnn30W69atQ2BgYGt3544SExODmJgYx/uBAwciIiIC77//PhYvXtyKPWt9drsdQUFB+OCDDyCXyxEVFYXz589j2bJlkgk1tyotLQ2RkZEYMGBAa3fljnDgwAEsWbIEq1evRnR0NAoKCjB79mwsXrwY8+bNa5Z9MtTcYQIDAyGXy1FcXOxUXlxcjODg4Btuu3z5cixduhR79+5Fr169mrObLa6p50Umk6Fr164AgD59+iAvLw8pKSmSCjXunpuTJ0/izJkzGDlypKPMbrcDADw8PJCfn48uXbo0b6dbwK38XaqlUCjQt29fFBQUNEcXW01Tzk1ISAgUCgXkcrmjLCIiAnq9HmazGUqlsln73BJu5TNjNBqRnp6ORYsWNWcXW01Tzs28efPw7LPP4vnnnwcAREZGwmg0Ytq0aXjjjTcgk93+O2B4T80dRqlUIioqCllZWY4yu92OrKwsp/9BXu+vf/0rFi9ejMzMTPTr168lutqimnperme322EymZqji63G3XNz//3344cffsDRo0cdy5/+9CcMHToUR48ehVarbcnuN5vb8Zmx2Wz44YcfEBIS0lzdbBVNOTeDBg1CQUGBIwADwIkTJxASEiKJQAPc2mdm69atMJlMeOaZZ5q7m62iKeemsrKyXnCpDcVic/3sZLPdgkxNlp6eLqpUKnHjxo3iTz/9JE6bNk0MCAhwTC199tlnxblz5zrqL126VFQqleJnn33mNLXwypUrrXUIzcLd87JkyRJx9+7d4smTJ8WffvpJXL58uejh4SGuW7eutQ6h2bh7bq4n1dlP7p6XhQsXirt27RJPnjwp5ubmiuPHjxfVarX4448/ttYhNBt3z01hYaHo6+srzpw5U8zPzxe//PJLMSgoSHzrrbda6xCaRVP/Lg0ePFgcN25cS3e3Rbl7bpKTk0VfX1/x008/FU+dOiXu3r1b7NKlizh27Nhm6yNDzR3qvffeEzt06CAqlUpxwIAB4rfffutY9/DDD4vPPfec433Hjh1FAPWW5OTklu94M3PnvLzxxhti165dRbVaLbZp00aMiYkR09PTW6HXLcOdc3M9qYYaUXTvvLz88suOuhqNRhwxYoR45MiRVuh1y3D3M/PNN9+I0dHRokqlEjt37iz+5S9/Ea1Wawv3uvm5e16OHz8uAhB3797dwj1tee6cG4vFIi5YsEDs0qWLqFarRa1WK86YMUP89ddfm61/gig21xgQERERUcvhPTVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJDDVEREQkCQw1REREJAkMNURERCQJ/x/ltAgOamtJdwAAAABJRU5ErkJggg==\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "plot(training_logs.logs, [\"eval_accuracy\"], [\"Evaluation accuracy\"])"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 450
        },
        "id": "7cfERwdudGXy",
        "outputId": "c397856d-4430-4994-c9ea-abdd34b119cb"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 640x480 with 1 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiwAAAGdCAYAAAAxCSikAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjcuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/bCgiHAAAACXBIWXMAAA9hAAAPYQGoP6dpAAA750lEQVR4nO3deXgUZb728bvTSXcWskHISiSEQAQh4IDEsKiM0SCKy3gUlYOAI46KHjR6FFRAlgFUxuGoKCPDpkeFOQ4oKoNLFMcF5R00I2swIewkrNnJ1l3vH0hLkwToQNJF+H6uq6/Q1U9V/+qhoe889VSVxTAMQwAAACbm4+0CAAAATofAAgAATI/AAgAATI/AAgAATI/AAgAATI/AAgAATI/AAgAATI/AAgAATM/X2wWcC06nU3v37lVwcLAsFou3ywEAAGfAMAyVlpYqNjZWPj6nHkNpEYFl7969io+P93YZAACgEXbt2qV27dqdsk2LCCzBwcGSju1wSEiIl6sBAABnoqSkRPHx8a7v8VNpEYHl+GGgkJAQAgsAAOeZM5nOwaRbAABgegQWAABgegQWAABgei1iDgsAnE8Mw1Btba0cDoe3SwGanNVqla+v71lfdoTAAgDNqLq6Wvv27VNFRYW3SwGaTWBgoGJiYmSz2Rq9DQILADQTp9Op/Px8Wa1WxcbGymazcbFLtGiGYai6uloHDhxQfn6+OnXqdNoLxDWEwAIAzaS6ulpOp1Px8fEKDAz0djlAswgICJCfn5927Nih6upq+fv7N2o7jYo5c+bMUUJCgvz9/ZWamqq1a9eesn1RUZHGjBmjmJgY2e12de7cWStXrnS9/uyzz8pisbg9Lr744saUBgCm19jfMIHz1bn4zHs8wrJ06VJlZmZq7ty5Sk1N1ezZs5WRkaGcnBxFRkbWaV9dXa1rrrlGkZGRevfddxUXF6cdO3YoLCzMrd0ll1yizz777NfCfBn8AQAAx3gceV588UWNHj1ao0aNUteuXTV37lwFBgZqwYIF9bZfsGCBDh8+rPfee0/9+vVTQkKCrrzySvXo0cOtna+vr6Kjo12PiIiIxu0RAOC8s337dlksFmVnZzf5ey1atKjOL80wP48CS3V1tdatW6f09PRfN+Djo/T0dK1Zs6bedVasWKG0tDSNGTNGUVFR6tatm6ZPn17ndL6ff/5ZsbGxSkxM1LBhw7Rz584G66iqqlJJSYnbAwDQNEaOHFnnsL3FYtGgQYO8XdppJSQkaPbs2W7Lhg4dqq1bt3qnIDSaR8ddDh48KIfDoaioKLflUVFR2rJlS73rbNu2TZ9//rmGDRumlStXKjc3Vw8++KBqamo0adIkSVJqaqoWLVqk5ORk7du3T5MnT9aAAQO0YcOGem+INGPGDE2ePNmT0gEAZ2HQoEFauHCh2zK73e6las5OQECAAgICvF1GszMMQw6H47ydctHkM7+cTqciIyP1+uuvq1evXho6dKiefvppzZ0719Xmuuuu02233aaUlBRlZGRo5cqVKioq0t/+9rd6tzl+/HgVFxe7Hrt27Wrq3QCAC5rdbnc7bB8dHa3w8HBJ0l133aWhQ4e6ta+pqVFERITeeOMNSdKqVavUv39/hYWFqU2bNrrhhhuUl5fX4PvVd9jmvffeczsNPC8vTzfddJOioqLUqlUrXXbZZW5zIa+66irt2LFDjz76qGtUqKFtv/baa+rYsaNsNpuSk5P15ptvur1usVj017/+VbfccosCAwPVqVMnrVix4pR99uabb6p3794KDg5WdHS07rrrLu3fv9+tzcaNG3XDDTcoJCREwcHBGjBggFu/LFiwQJdcconsdrtiYmL00EMPSar/EFpRUZEsFotWr14tSVq9erUsFov+8Y9/qFevXrLb7fr6669P22/SsSMZTz75pOLj42W325WUlKT58+fLMAwlJSVp1qxZbu2zs7NlsViUm5t7yj45Gx4FloiICFmtVhUWFrotLywsVHR0dL3rxMTEqHPnzrJara5lXbp0UUFBgaqrq+tdJywsTJ07d25wx+12u+vOzNyhGcD5yjAMVVTXeuVhGMY5249hw4bpgw8+UFlZmWvZxx9/rIqKCt1yyy2SpPLycmVmZupf//qXsrKy5OPjo1tuuUVOp7PR71tWVqbBgwcrKytLP/74owYNGqQhQ4a4phQsW7ZM7dq105QpU7Rv3z7t27ev3u0sX75cY8eO1WOPPaYNGzboD3/4g0aNGqUvvvjCrd3kyZN1++2366efftLgwYM1bNgwHT58uMH6ampqNHXqVP373//We++9p+3bt2vkyJGu1/fs2aMrrrhCdrtdn3/+udatW6d77rlHtbW1ko6FqDFjxui+++7T+vXrtWLFCiUlJXncT+PGjdPMmTO1efNmpaSknLbfJOnuu+/WO++8o5deekmbN2/WX/7yF7Vq1UoWi0X33HNPndG2hQsX6oorrmhUfWfKo3Ehm82mXr16KSsrSzfffLOkYyMoWVlZrtR3sn79+untt9+W0+l0nda0devWU17xrqysTHl5eRo+fLgn5QHAeeVojUNdJ37slffeNCVDgbYz/wr48MMP1apVK7dlTz31lJ566illZGQoKChIy5cvd/2//fbbb+vGG290Hda/9dZb3dZdsGCB2rZtq02bNqlbt26N2ocePXq4ncAxdepULV++XCtWrNBDDz2k1q1by2q1ukY4GjJr1iyNHDlSDz74oCQpMzNT3333nWbNmqWBAwe62o0cOVJ33nmnJGn69Ol66aWXtHbt2gbn8txzzz2uPycmJuqll17SZZddprKyMrVq1Upz5sxRaGiolixZIj8/P0lS586dXetMmzZNjz32mMaOHetadtlll3nSRZKkKVOm6JprrnE9b9269Sn7bevWrfrb3/6mTz/91DVnNTEx0a0fJk6cqLVr16pPnz6qqanR22+/XWfU5Vzz+JBQZmam5s2bp8WLF2vz5s164IEHVF5erlGjRkk6lsrGjx/vav/AAw/o8OHDGjt2rLZu3aqPPvpI06dP15gxY1xtHn/8cX355Zfavn27vv32W91yyy2yWq2uDwYAwLsGDhyo7Oxst8f9998v6dhZnrfffrveeustScdGU95//30NGzbMtf7PP/+sO++8U4mJiQoJCVFCQoIknfIEi9MpKyvT448/ri5duigsLEytWrXS5s2bPd7m5s2b1a9fP7dl/fr10+bNm92WpaSkuP4cFBSkkJCQOod4TrRu3ToNGTJEF110kYKDg3XllVdK+nWfs7OzNWDAAFdYOdH+/fu1d+9eXX311R7tS3169+7t9vx0/ZadnS2r1eqq92SxsbG6/vrrXWcHf/DBB6qqqtJtt9121rWeisczb4YOHaoDBw5o4sSJKigoUM+ePbVq1SrXRNydO3e6XSAmPj5eH3/8sR599FGlpKQoLi5OY8eO1ZNPPulqs3v3bt155506dOiQ2rZtq/79++u7775T27Ztz8EuAoA5BfhZtWlKhtfe2xNBQUGnHO4fNmyYrrzySu3fv1+ffvqpAgIC3EYehgwZovbt22vevHmKjY2V0+lUt27dGpwa4OPjU+ewVU1Njdvzxx9/XJ9++qlmzZqlpKQkBQQE6D/+4z8a3ObZOjlYWCyWBg9plZeXKyMjQxkZGXrrrbfUtm1b7dy5UxkZGa76TjXx93STgo9/z57YRyf3z3FBQUFuz0/Xb2cyIfnee+/V8OHD9ec//1kLFy7U0KFDm/zqzY2aKvzQQw81eAjo+GSfE6Wlpem7775rcHtLlixpTBkAcF6zWCweHZYxs759+yo+Pl5Lly7VP/7xD912222uL/hDhw4pJydH8+bN04ABAyRJX3/99Sm317ZtW5WWlqq8vNz1hXvyNVq++eYbjRw50jVPpqysTNu3b3drY7PZTntX7C5duuibb77RiBEj3LbdtWvX0+53Q7Zs2aJDhw5p5syZio+PlyT961//cmuTkpKixYsXq6ampk4YCg4OVkJCgrKystwOSx13/Bf6ffv26dJLL5VUt38acrp+6969u5xOp7788ku3y5icaPDgwQoKCtJrr72mVatW6Z///OcZvffZaBn/UgAATaqqqkoFBQVuy3x9fd0u8nnXXXdp7ty52rp1q9uE1fDwcLVp00avv/66YmJitHPnTo0bN+6U75eamqrAwEA99dRT+q//+i99//33WrRokVubTp06admyZRoyZIgsFosmTJhQZ8QjISFB//znP3XHHXfIbrfXe1HS//7v/9btt9+uSy+9VOnp6frggw+0bNmyOmfOeOKiiy6SzWbTyy+/rPvvv18bNmzQ1KlT3do89NBDevnll3XHHXdo/PjxCg0N1Xfffac+ffooOTlZzz77rO6//35FRkbquuuuU2lpqb755hs9/PDDCggI0OWXX66ZM2eqQ4cO2r9/v5555pkzqu10/ZaQkKARI0bonnvu0UsvvaQePXpox44d2r9/v26//XZJktVq1ciRIzV+/Hh16tRJaWlpje6rM2a0AMXFxYYko7i42NulAECDjh49amzatMk4evSot0vxyIgRIwxJdR7Jyclu7TZt2mRIMtq3b284nU631z799FOjS5cuht1uN1JSUozVq1cbkozly5cbhmEY+fn5hiTjxx9/dK2zfPlyIykpyQgICDBuuOEG4/XXXzdO/NrKz883Bg4caAQEBBjx8fHGK6+8Ylx55ZXG2LFjXW3WrFljpKSkGHa73bXuwoULjdDQULf6Xn31VSMxMdHw8/MzOnfubLzxxhtur59Y63GhoaHGwoULG+y3t99+20hISDDsdruRlpZmrFixos4+/vvf/zauvfZaIzAw0AgODjYGDBhg5OXluV6fO3eukZycbPj5+RkxMTHGww8/7NbfaWlpRkBAgNGzZ0/jk08+MSQZX3zxhWEYhvHFF18YkowjR4641XUm/Xb06FHj0UcfNWJiYgybzWYkJSUZCxYscNtOXl6eIcl4/vnnG+yDE7dX32ffk+9vi2Gcw3PbvKSkpEShoaEqLi7mFGcAplVZWan8/Hx16NCh0XesBcziq6++0tVXX61du3bVuaDsyRr67Hvy/c0hIQAAcMaqqqp04MABPfvss7rttttOG1bOFe5xDgAAztg777yj9u3bq6ioSM8//3yzvS+BBQAAnLGRI0fK4XBo3bp1iouLa7b3JbAAAADTI7AAAADTI7AAQDNrASdnAh45F595AgsANJPjVzOtqKjwciVA8zr+ma/vvklnitOaAaCZWK1WhYWFuW6YFxgYKIvF4uWqgKZjGIYqKiq0f/9+hYWFyWr17B5WJyKwAEAzio6OlqRT3uUXaGnCwsJcn/3GIrAAQDOyWCyKiYlRZGRkg3fXBVoSPz+/sxpZOY7AAgBeYLVaz8l/4sCFgkm3AADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9AgsAADA9BoVWObMmaOEhAT5+/srNTVVa9euPWX7oqIijRkzRjExMbLb7ercubNWrlx5VtsEAAAXDo8Dy9KlS5WZmalJkybphx9+UI8ePZSRkaH9+/fX2766ulrXXHONtm/frnfffVc5OTmaN2+e4uLiGr1NAABwYbEYhmF4skJqaqouu+wyvfLKK5Ikp9Op+Ph4Pfzwwxo3blyd9nPnztULL7ygLVu2yM/P75xs82QlJSUKDQ1VcXGxQkJCPNkdAADgJZ58f3s0wlJdXa1169YpPT391w34+Cg9PV1r1qypd50VK1YoLS1NY8aMUVRUlLp166bp06fL4XA0eptVVVUqKSlxewAAgJbLo8By8OBBORwORUVFuS2PiopSQUFBvets27ZN7777rhwOh1auXKkJEyboT3/6k6ZNm9bobc6YMUOhoaGuR3x8vCe7AQAAzjNNfpaQ0+lUZGSkXn/9dfXq1UtDhw7V008/rblz5zZ6m+PHj1dxcbHrsWvXrnNYMQAAMBtfTxpHRETIarWqsLDQbXlhYaGio6PrXScmJkZ+fn6yWq2uZV26dFFBQYGqq6sbtU273S673e5J6QAA4Dzm0QiLzWZTr169lJWV5VrmdDqVlZWltLS0etfp16+fcnNz5XQ6Xcu2bt2qmJgY2Wy2Rm0TAABcWDw+JJSZmal58+Zp8eLF2rx5sx544AGVl5dr1KhRkqS7775b48ePd7V/4IEHdPjwYY0dO1Zbt27VRx99pOnTp2vMmDFnvE0AAHBh8+iQkCQNHTpUBw4c0MSJE1VQUKCePXtq1apVrkmzO3fulI/PrzkoPj5eH3/8sR599FGlpKQoLi5OY8eO1ZNPPnnG2wQAABc2j6/DYkZchwUAgPNPk12HBQAAwBsILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAwPQILAAAoEFOp6G3v9+p+V/ne7UOX6++OwAAMK28A2Uav2y91uYfls3XR1dfHKmEiCCv1EJgAQAAbmocTr3+z236n6yfVV3rVICfVY9nJCu+daDXaiKwAAAAl+xdRRr395+0paBUkjSgU4Sm39Ldq2FFIrAAAABJ5VW1+tMnW7Xo23w5DSk80E8Th3TVzT3jZLFYvF0egQUAgAvdl1sP6Kll67Wn6Kgk6eaesZpwQ1e1aWX3cmW/IrAAAHCBOlxerWkfbtKyH/dIkuLCAjTtlm4amBzp5crqIrAAAHCBMQxD72fv1ZQPN+lwebUsFmlk3wQ9fm2yguzmjAbmrAoAADSJ3Ucq9PTyDfpy6wFJUnJUsGbe2l2XXhTu5cpOjcACAMAFwOE0tPjb7Zr1SY4qqh2yWX308G+T9IcrO8rma/7ryBJYAABo4bYUlOjJv6/Xv3cVSZIuSwjXjN+lKCmylXcL8wCBBQCAFqqyxqE5X+TqtdV5qnUaamX31bjrLtZdfS6Sj4/3T1X2RKPGgObMmaOEhAT5+/srNTVVa9eubbDtokWLZLFY3B7+/v5ubUaOHFmnzaBBgxpTGgAAkLQ2/7AGv/SVXv48V7VOQ9d0jdJnmVfqPy9vf96FFakRIyxLly5VZmam5s6dq9TUVM2ePVsZGRnKyclRZGT9p0GFhIQoJyfH9by+C9AMGjRICxcudD23281z7jcAAOeLksoaPfePLXrr+52SpLbBdk258RIN6hZtigvANZbHgeXFF1/U6NGjNWrUKEnS3Llz9dFHH2nBggUaN25cvetYLBZFR0efcrt2u/20bQAAQMM+2VigCe9vUGFJlSRpaO94PTW4i0ID/bxc2dnz6JBQdXW11q1bp/T09F834OOj9PR0rVmzpsH1ysrK1L59e8XHx+umm27Sxo0b67RZvXq1IiMjlZycrAceeECHDh1qcHtVVVUqKSlxewAAcKHaX1qpB99ap/veXKfCkioltAnU26NT9dx/pLSIsCJ5GFgOHjwoh8OhqKgot+VRUVEqKCiod53k5GQtWLBA77//vv73f/9XTqdTffv21e7du11tBg0apDfeeENZWVl67rnn9OWXX+q6666Tw+God5szZsxQaGio6xEfH+/JbgAA0CIYhqGl/2+n0v/0pVauL5DVx6IHruqoVY9cob4dI7xd3jllMQzDONPGe/fuVVxcnL799lulpaW5lj/xxBP68ssv9f333592GzU1NerSpYvuvPNOTZ06td4227ZtU8eOHfXZZ5/p6quvrvN6VVWVqqqqXM9LSkoUHx+v4uJihYSEnOnuAABw3so/WK6nlq3Xmm3Hjkh0jwvVzFu765LYUC9XduZKSkoUGhp6Rt/fHs1hiYiIkNVqVWFhodvywsLCM55/4ufnp0svvVS5ubkNtklMTFRERIRyc3PrDSx2u51JuQCAC1KNw6l5X23T/3z2s6pqnfL389Fj1yRrVL8E+VrNfwG4xvJoz2w2m3r16qWsrCzXMqfTqaysLLcRl1NxOBxav369YmJiGmyze/duHTp06JRtAAC40KzfXaybXvlGz6/KUVWtU/2TIvTJI1dq9BWJLTqsSI04SygzM1MjRoxQ79691adPH82ePVvl5eWus4buvvtuxcXFacaMGZKkKVOm6PLLL1dSUpKKior0wgsvaMeOHbr33nslHZuQO3nyZN16662Kjo5WXl6ennjiCSUlJSkjI+Mc7ioAAOeno9UO/fmzrfrrV9vkNKTQAD9NuKGrbv1N3Hl9qrInPA4sQ4cO1YEDBzRx4kQVFBSoZ8+eWrVqlWsi7s6dO+Xj82vKO3LkiEaPHq2CggKFh4erV69e+vbbb9W1a1dJktVq1U8//aTFixerqKhIsbGxuvbaazV16lQO+wAALnhf/XxATy1fr12Hj0qShvSI1cQbuqpt8IX1HenRpFuz8mTSDgAA54Mj5dWa9tFm/f2HY2fVxoT6a9rN3XR1l6jTrHn+aLJJtwAAoGkZhqEPftqnKR9s1MGyalks0t2Xt9d/D7pYrewX7tf2hbvnAACYzJ6io5rw3gZ9vmW/JKlTZCvNvDVFvdqHe7ky7yOwAADgZQ6nof/9boeeX7VF5dUO+VktemhgJ91/VaLsvlZvl2cKBBYAALxoa2Gpxv39J/2ws0iS1Kt9uGb+rrs6RQV7tzCTIbAAAOAFVbUOvfpFnl5dnasah6Egm1XjrrtYw1Lby8fnwjhV2RMEFgAAmtm6HYf15N/XK3d/mSTp6osjNfXmbooNC/ByZeZFYAEAoJmUVtbohY9z9OZ3O2QYUkQrm5698RJd3z3mgrkAXGMRWAAAaAZZmwv1zHsbtK+4UpJ0W692evr6LgoLtHm5svMDgQUAgCZ0oLRKkz/YqA9/2idJuqh1oKbf0l39O0V4ubLzC4EFAIAmYBiG/m/dbv3xo80qPlojH4s0ekCiHknvrAAbpyp7isACAMA5tuNQuZ5avl7f5B6SJF0SG6Lnbk1Rt7hQL1d2/iKwAABwjtQ6nFrwTb5e/HSrKmucsvv66NFrOuve/h3ka/U5/QbQIAILAADnwIY9xRq37Cdt2FMiSUpLbKMZv+uuhIggL1fWMhBYAAA4C5U1Dv35s63661f5cjgNhfj76pnru+q23u04VfkcIrAAANBI3+Ye1Pjl67XjUIUk6fqUGE0a0lWRwf5erqzlIbAAAOCh4ooa/XHlJv3tX7slSdEh/pp6czdd0zXKy5W1XAQWAADOkGEYWrm+QJNWbNTBsipJ0vDL2+uJQckK9vfzcnUtG4EFAIAzsK/4qCa8t1GfbS6UJHVsG6SZt6bosoTWXq7swkBgAQDgFJxOQ2+t3ann/rFFZVW18rNa9MBVSRozsKPsvlwArrkQWAAAaEDu/lKN+/t6/WvHEUlSz/gwPXdripKjg71c2YWHwAIAwEmqa52a+2WeXvk8V9UOpwJtVj2RkazhaQmy+nCqsjcQWAAAOMEPO49o3N9/0tbCMknSwOS2mnZLd8WFBXi5sgsbgQUAAEllVbWa9XGOFq/ZLsOQ2gTZNHFIV93YI5YLwJkAgQUAcMH7Yst+PfPeBu0pOipJ+t1v4jTh+q4KD7J5uTIcR2ABAFywDpVVafIHm7Ti33slSe3CAzT9lu66onNbL1eGkxFYAAAXHMMwtPzHPZr64SYdqaiRj0X6ff8OevSazgq08dVoRvytAAAuKLsOV+ip5ev11c8HJUldYkL03K3dldIuzLuF4ZQILACAC0Ktw6lF327Xnz7ZqqM1Dtl8ffRIeieNHpAoP6uPt8vDaRBYAAAt3qa9JRq37Cf9tLtYkpTaobVm/K67Etu28nJlOFMEFgBAi1VZ49BLWT/r9X9uU63TULC/r54e3EW3946XDxeAO68QWAAALdJ32w5p/LL1yj9YLkm6rlu0Jt94iSJD/L1cGRqDwAIAaFGKj9Zo5j826521uyRJkcF2TbmpmwZ1i/ZyZTgbBBYAQIuxasM+TXx/o/aXVkmS7kq9SE8OulihAX5ergxni8ACADjvFZZUauL7G/TxxkJJUmJEkGb8rrtSE9t4uTKcKwQWAMB5y+k0tOT/7dKMlZtVWlUrXx+L7r+yox76bZL8/azeLg/nEIEFAHBeyjtQpvHL1mtt/mFJUo92oZp5a4q6xIR4uTI0BQILAOC8UuNw6vV/btP/ZP2s6lqnAvysejwjWSP7JsjKqcotFoEFAHDeyN5VpHF//0lbCkolSVd0bqs/3txN8a0DvVwZmhqBBQBgehXVtfrTJ1u18Jt8OQ0pPNBPk4Zcopt6xspiYVTlQkBgAQCY2pdbD+jp5eu1+8hRSdItl8bpmeu7qE0ru5crQ3MisAAATOlwebWmfbhJy37cI0mKCwvQH2/ppquSI71cGbyBwAIAMBXDMPR+9l5N+XCTDpdXy2KRRvXtoMeu7awgO19bFyr+5k/jnbU7FWizKsjmqyC7r1rZfRVkt/7y01eBNivHTwHgHNl9pEJPL9+gL7cekCQlRwVr5q3ddelF4V6uDN5GYDmFGodT45etP2Ubi0W/hBnrr4HGFW5OWGZ3X3ZiW1cI8veV3ZcLHQG48DichhZ/u12zPslRRbVDNquP/uvqJN13RUfZfH28XR5MgMByCrUOQ9d2jVJ5da3KKmtVVlWr8iqHyqtqVV5dK6chGYZUVnXsNanqrN/Tz2o5FmhOCDInhp46yxoY+Tm2Dat8rfxDB2BuWwpK9OTf1+vfu4okSX0SWmv677orKbKVdwuDqRBYTiHAZtXrd/eu9zXDMHS0xuEWYo79ubbeZeXVtSo7eVnVr8uO1jgkSTUOQ0UVNSqqqDkn++Dv5/NriGkgBNUZDeLwF4BmUFnj0JwvcvXa6jzVOg0F2301bvDFuvOyi+TDBeBwEgJLI1ksFgXafBVo85WCz357Dqeh8uq6Qaa+cFNfCDq+/PhrNQ5DklRZ41RlTbUOllWfdY3n8vBXkN1Xdl8fAhBwgVqbf1jjlv2kbQfKJUnXdI3S1Ju6KTrU38uVwawILCZh9bEoxN9PIf7n5hboVbWOMxr5KXOFJMdJ4eiEttW1Mprg8Jevj+U0h7lOCjwNLuPwF3C+KKms0XP/2KK3vt8pSWobbNeUGy/RoG7R/AKDUyKwtFB2X6vsvla1DrKd9baa6vBXrdNQ8dEaFR9t+sNf0SH+uqhNoOJbB+qi1oFqFx7ABGegmX2ysUAT3t+gwpJjv/Dc2Sde4wZ1UWjguflFDS0bgQWn1ZSHv06czOweeOqO/NRZVt34w18WixQd4u8KMMcfx59HtLLx2x5wlgzD0KHyau0+clSv/zNPK9cXSJI6RARp+i3dldaxjZcrxPmEwIJm53b4K/Tst3e6w18llTXaV1ypnYcrtOtwhXYerlBFtUP7iiu1r7jSdWv6EwX4Wd0CzEWtA9S+TZDifxmd8fdjdAaodThVWFqlPUeOak9RxS8/j2r3Lz/3Fh1VZY3T1d7qY9EfrkjUf13diX9D8BiBBec9Tw9/Hf+tzxVgDh0LMcef7yup1NEah3IKS5VTWFrvNqJD/N0DTZsA1/O2reyMzqBFqKxxaE/RUVcQ2fvLn3f/8rOgpFIOp3HKbVgsUmSwXRdHh+iJQcm6JPYc/JaCC5LFMIxTf9rOAyUlJQoNDVVxcbFCQkK8XQ7Oc1W1Du05ctRtRObY46h2HipXebXjlOv7+/nUOcR04nN+s4RZFB+tcYWRPUcqjv08IaCcyeFVP6tFMaEBigsLUFz4rz/b/fIzOtSf+WJokCff34ywACex+1qV2LaVEtvWvWiVYRg6UlHjNiJz4gjNvuJjQ+BbC8u0tbCs3u1HBtvrBpo2x362bWXn+hM4J5xOQwfLqlyjIXtPCiN7jhxVaVXtabcTZLMqLjxAsWHuoaRdeIDiwgLVNtguK59ZNAMCC+ABi8Wi1kE2tQ6yqWd8WJ3Xq2ud2lt01D3QHH8cqlBpVa32l1Zpf2mV/rXjSJ317b4+bqMy7n8OODbxGdCxW4cUFFe65ou45pEcDyjFlaqudZ52O62DbMeCyEkjJMdDSWiAH4c4YQr87wecQzZfHyVEBCkhIqjOa4Zx7DTunYcr6g00e4sqVVXrVO7+MuXur390JqKVXe3b1H+4KTKY0ZmWpKK61m2+yIlzSPYUHVVhSaVOM31EPr+cDRfbQBiJDSME4/zBJxVoJhaLRWGBNoUF2pTSLqzO6zUOp/YVHTubacfhcrdAs+NQhUora3WwrEoHy6q0rp7RGZuvj+LDA+o93BQfHqggO//czcIwjt2C48Qzak4eITlyBrfnsPn6/Do6Uk8oiQ71lx8XVEQLwf9ggEn4WX2OzWVpE6j+iqjzenGF++jMiYFmT9FRVdc6lXegXHm/XOr8ZBGtbA0ebooO8Wd05hxyOA3tL62sc5rviXNJKk4zeVuSgu2+dULIiT8jghhVw4WDs4SAFqDW4XRda6a+QHO6m2narD5qFx5Qf6BpE6hWjM64qap1aF9RpSuE7D5phKSguNJ1QcNTiWhldzujxm2kJDzgnN2qAzArzhICLjC+1mOTdeNbB6pfPa8XH63RrjqnaR97vvvIUVU7nNp2sFzbDtY/OtM6yOZ+Eb3Wxy6id1GbY6MzLe0skdLKmjpn1Jw4l+RA6envpWX1sSg6xL9uIPnlZ2wYFyAEPNGowDJnzhy98MILKigoUI8ePfTyyy+rT58+9bZdtGiRRo0a5bbMbrersrLS9dwwDE2aNEnz5s1TUVGR+vXrp9dee02dOnVqTHkAThIa4KfQuFB1i6t70a5ah1MFJZUnTQL+9To0h8urXY9/7yqqs76f1aJ24cdHZOrOoQk22SjB8QsH7jnifprv7hOuR1JSefrTff39fpk/Eh54wmm+vwaSyGA7N+QEziGPA8vSpUuVmZmpuXPnKjU1VbNnz1ZGRoZycnIUGRlZ7zohISHKyclxPT/5FLnnn39eL730khYvXqwOHTpowoQJysjI0KZNm+Tvz63Ggabka/VRu/BAtQsPlDrWfb20ska7Dte9kN6uwxXadaRCNQ5D+QfLld/A6Ex4oF+DF9GLCfU/51/qxwPYyYHkxD9XncHpvqEBfvVcd+TX562DuN8U0Jw8nsOSmpqqyy67TK+88ookyel0Kj4+Xg8//LDGjRtXp/2iRYv0yCOPqKioqN7tGYah2NhYPfbYY3r88cclScXFxYqKitKiRYt0xx13nLYm5rAA3uFwGsdGZw7Vf7jpUPmpr5Tq62NRXHhAg4EmNKDu6ExljUO7G7gQ2p4izy4Xf+IIyYmHbmLDApi3AzSDJpvDUl1drXXr1mn8+PGuZT4+PkpPT9eaNWsaXK+srEzt27eX0+nUb37zG02fPl2XXHKJJCk/P18FBQVKT093tQ8NDVVqaqrWrFlTb2CpqqpSVdWvx5BLSko82Q0A54jVx+KaKFrfnXfLqmpdQebkQLP78LG5MzsOHTttuz6hAX6ua8wcKDt2k73ThSCJy8UDLZFHgeXgwYNyOByKiopyWx4VFaUtW7bUu05ycrIWLFiglJQUFRcXa9asWerbt682btyodu3aqaCgwLWNk7d5/LWTzZgxQ5MnT/akdABe0Mruqy4xIeoSU/c3J6fTUGFppevWBifft+lgWZWKj9Zo/Z7iOusev1z8r4Ek8Jef/ooL4yJ6QEvU5GOeaWlpSktLcz3v27evunTpor/85S+aOnVqo7Y5fvx4ZWZmup6XlJQoPj7+rGsF0Hx8fI6NgsSEBig1se7oTEV1rWvuzP7SymOnAHO5eOCC5VFgiYiIkNVqVWFhodvywsJCRUdHn9E2/Pz8dOmllyo3N1eSXOsVFhYqJibGbZs9e/asdxt2u112u92T0gGcZwJtvkqODlZydLC3SwFgAh5Nz7fZbOrVq5eysrJcy5xOp7KystxGUU7F4XBo/fr1rnDSoUMHRUdHu22zpKRE33///RlvEwAAtGweHxLKzMzUiBEj1Lt3b/Xp00ezZ89WeXm561ord999t+Li4jRjxgxJ0pQpU3T55ZcrKSlJRUVFeuGFF7Rjxw7de++9ko6d4vzII49o2rRp6tSpk+u05tjYWN18883nbk8BAMB5y+PAMnToUB04cEATJ05UQUGBevbsqVWrVrkmze7cuVM+Pr8O3Bw5ckSjR49WQUGBwsPD1atXL3377bfq2rWrq80TTzyh8vJy3XfffSoqKlL//v21atUqrsECAAAkcS8hAADgJZ58f3PdaAAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHoEFgAAYHqNCixz5sxRQkKC/P39lZqaqrVr157RekuWLJHFYtHNN9/stnzkyJGyWCxuj0GDBjWmNAAA0AJ5HFiWLl2qzMxMTZo0ST/88IN69OihjIwM7d+//5Trbd++XY8//rgGDBhQ7+uDBg3Svn37XI933nnH09IAAEAL5XFgefHFFzV69GiNGjVKXbt21dy5cxUYGKgFCxY0uI7D4dCwYcM0efJkJSYm1tvGbrcrOjra9QgPD/e0NAAA0EJ5FFiqq6u1bt06paen/7oBHx+lp6drzZo1Da43ZcoURUZG6ve//32DbVavXq3IyEglJyfrgQce0KFDhxpsW1VVpZKSErcHAABouTwKLAcPHpTD4VBUVJTb8qioKBUUFNS7ztdff6358+dr3rx5DW530KBBeuONN5SVlaXnnntOX375pa677jo5HI5628+YMUOhoaGuR3x8vCe7AQAAzjO+Tbnx0tJSDR8+XPPmzVNERESD7e644w7Xn7t3766UlBR17NhRq1ev1tVXX12n/fjx45WZmel6XlJSQmgBAKAF8yiwREREyGq1qrCw0G15YWGhoqOj67TPy8vT9u3bNWTIENcyp9N57I19fZWTk6OOHTvWWS8xMVERERHKzc2tN7DY7XbZ7XZPSgcAAOcxjw4J2Ww29erVS1lZWa5lTqdTWVlZSktLq9P+4osv1vr165Wdne163HjjjRo4cKCys7MbHBXZvXu3Dh06pJiYGA93BwAAtEQeHxLKzMzUiBEj1Lt3b/Xp00ezZ89WeXm5Ro0aJUm6++67FRcXpxkzZsjf31/dunVzWz8sLEySXMvLyso0efJk3XrrrYqOjlZeXp6eeOIJJSUlKSMj4yx3DwAAtAQeB5ahQ4fqwIEDmjhxogoKCtSzZ0+tWrXKNRF3586d8vE584Ebq9Wqn376SYsXL1ZRUZFiY2N17bXXaurUqRz2AQAAkiSLYRiGt4s4WyUlJQoNDVVxcbFCQkK8XQ4AADgDnnx/cy8hAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgegQWAABgeo0KLHPmzFFCQoL8/f2VmpqqtWvXntF6S5YskcVi0c033+y23DAMTZw4UTExMQoICFB6erp+/vnnxpQGAABaII8Dy9KlS5WZmalJkybphx9+UI8ePZSRkaH9+/efcr3t27fr8ccf14ABA+q89vzzz+ull17S3Llz9f333ysoKEgZGRmqrKz0tDwAANACeRxYXnzxRY0ePVqjRo1S165dNXfuXAUGBmrBggUNruNwODRs2DBNnjxZiYmJbq8ZhqHZs2frmWee0U033aSUlBS98cYb2rt3r9577z2PdwgAALQ8vp40rq6u1rp16zR+/HjXMh8fH6Wnp2vNmjUNrjdlyhRFRkbq97//vb766iu31/Lz81VQUKD09HTXstDQUKWmpmrNmjW644476myvqqpKVVVVrufFxcWSpJKSEk92BwAAeNHx723DME7b1qPAcvDgQTkcDkVFRbktj4qK0pYtW+pd5+uvv9b8+fOVnZ1d7+sFBQWubZy8zeOvnWzGjBmaPHlyneXx8fGn2wUAAGAypaWlCg0NPWUbjwJLYwoYPny45s2bp4iIiHO23fHjxyszM9P13Ol06vDhw2rTpo0sFss5ex9vKCkpUXx8vHbt2qWQkBBvl2Mq9E396JeG0Tf1o1/qR780rKn6xjAMlZaWKjY29rRtPQosERERslqtKiwsdFteWFio6OjoOu3z8vK0fft2DRkyxLXM6XQee2NfX+Xk5LjWKywsVExMjNs2e/bsWW8ddrtddrvdbVlYWJgnu2J6ISEh/INpAH1TP/qlYfRN/eiX+tEvDWuKvjndyMpxHk26tdls6tWrl7KyslzLnE6nsrKylJaWVqf9xRdfrPXr1ys7O9v1uPHGGzVw4EBlZ2crPj5eHTp0UHR0tNs2S0pK9P3339e7TQAAcOHx+JBQZmamRowYod69e6tPnz6aPXu2ysvLNWrUKEnS3Xffrbi4OM2YMUP+/v7q1q2b2/rHR0JOXP7II49o2rRp6tSpkzp06KAJEyYoNja2zvVaAADAhcnjwDJ06FAdOHBAEydOVEFBgXr27KlVq1a5Js3u3LlTPj6enS39xBNPqLy8XPfdd5+KiorUv39/rVq1Sv7+/p6Wd96z2+2aNGlSnUNeoG8aQr80jL6pH/1SP/qlYWboG4txJucSAQAAeBH3EgIAAKZHYAEAAKZHYAEAAKZHYAEAAKZHYPGCOXPmKCEhQf7+/kpNTdXatWsbbDtv3jwNGDBA4eHhCg8PV3p6+inbn+886Ztly5apd+/eCgsLU1BQkHr27Kk333yzGattPp70y4mWLFkii8XSoi8R4EnfLFq0SBaLxe3RUs9G9PQzU1RUpDFjxigmJkZ2u12dO3fWypUrm6na5uNJv1x11VV1Pi8Wi0XXX399M1bcfDz9zMyePVvJyckKCAhQfHy8Hn30UVVWVjZdgQaa1ZIlSwybzWYsWLDA2LhxozF69GgjLCzMKCwsrLf9XXfdZcyZM8f48ccfjc2bNxsjR440QkNDjd27dzdz5U3P07754osvjGXLlhmbNm0ycnNzjdmzZxtWq9VYtWpVM1fetDztl+Py8/ONuLg4Y8CAAcZNN93UPMU2M0/7ZuHChUZISIixb98+16OgoKCZq256nvZLVVWV0bt3b2Pw4MHG119/beTn5xurV682srOzm7nypuVpvxw6dMjts7JhwwbDarUaCxcubN7Cm4GnffPWW28ZdrvdeOutt4z8/Hzj448/NmJiYoxHH320yWoksDSzPn36GGPGjHE9dzgcRmxsrDFjxowzWr+2ttYIDg42Fi9e3FQles3Z9o1hGMall15qPPPMM01Rntc0pl9qa2uNvn37Gn/961+NESNGtNjA4mnfLFy40AgNDW2m6rzH03557bXXjMTERKO6urq5SvSKs/0/5s9//rMRHBxslJWVNVWJXuNp34wZM8b47W9/67YsMzPT6NevX5PVyCGhZlRdXa1169YpPT3dtczHx0fp6elas2bNGW2joqJCNTU1at26dVOV6RVn2zeGYSgrK0s5OTm64oormrLUZtXYfpkyZYoiIyP1+9//vjnK9IrG9k1ZWZnat2+v+Ph43XTTTdq4cWNzlNtsGtMvK1asUFpamsaMGaOoqCh169ZN06dPl8PhaK6ym9y5+P93/vz5uuOOOxQUFNRUZXpFY/qmb9++Wrduneuw0bZt27Ry5UoNHjy4yeps0rs1w93BgwflcDhcVwU+LioqSlu2bDmjbTz55JOKjY11+2C1BI3tm+LiYsXFxamqqkpWq1WvvvqqrrnmmqYut9k0pl++/vprzZ8/X9nZ2c1Qofc0pm+Sk5O1YMECpaSkqLi4WLNmzVLfvn21ceNGtWvXrjnKbnKN6Zdt27bp888/17Bhw7Ry5Url5ubqwQcfVE1NjSZNmtQcZTe5s/3/d+3atdqwYYPmz5/fVCV6TWP65q677tLBgwfVv39/GYah2tpa3X///XrqqaearE4Cy3lk5syZWrJkiVavXt1iJwp6Kjg4WNnZ2SorK1NWVpYyMzOVmJioq666ytuleUVpaamGDx+uefPmKSIiwtvlmE5aWprbTVX79u2rLl266C9/+YumTp3qxcq8y+l0KjIyUq+//rqsVqt69eqlPXv26IUXXmgxgeVszZ8/X927d1efPn28XYoprF69WtOnT9err76q1NRU5ebmauzYsZo6daomTJjQJO9JYGlGERERslqtKiwsdFteWFio6OjoU647a9YszZw5U5999plSUlKaskyvaGzf+Pj4KCkpSZLUs2dPbd68WTNmzGgxgcXTfsnLy9P27ds1ZMgQ1zKn0ylJ8vX1VU5Ojjp27Ni0RTeTs/n3dJyfn58uvfRS5ebmNkWJXtGYfomJiZGfn5+sVqtrWZcuXVRQUKDq6mrZbLYmrbk5nM3npby8XEuWLNGUKVOaskSvaUzfTJgwQcOHD9e9994rSerevbvrnoBPP/20x/cUPBPMYWlGNptNvXr1UlZWlmuZ0+lUVlaW2299J3v++ec1depUrVq1Sr17926OUptdY/vmZE6nU1VVVU1Rold42i8XX3yx1q9fr+zsbNfjxhtv1MCBA5Wdna34+PjmLL9JnYvPjMPh0Pr16xUTE9NUZTa7xvRLv379lJub6wq3krR161bFxMS0iLAind3n5f/+7/9UVVWl//zP/2zqMr2iMX1TUVFRJ5QcD7xGU92isMmm86JeS5YsMex2u7Fo0SJj06ZNxn333WeEhYW5Tq0cPny4MW7cOFf7mTNnGjabzXj33XfdTq8rLS311i40GU/7Zvr06cYnn3xi5OXlGZs2bTJmzZpl+Pr6GvPmzfPWLjQJT/vlZC35LCFP+2by5MnGxx9/bOTl5Rnr1q0z7rjjDsPf39/YuHGjt3ahSXjaLzt37jSCg4ONhx56yMjJyTE+/PBDIzIy0pg2bZq3dqFJNPbfUv/+/Y2hQ4c2d7nNytO+mTRpkhEcHGy88847xrZt24xPPvnE6Nixo3H77bc3WY0EFi94+eWXjYsuusiw2WxGnz59jO+++8712pVXXmmMGDHC9bx9+/aGpDqPSZMmNX/hzcCTvnn66aeNpKQkw9/f3wgPDzfS0tKMJUuWeKHqpudJv5ysJQcWw/Csbx555BFX26ioKGPw4MHGDz/84IWqm56nn5lvv/3WSE1NNex2u5GYmGj88Y9/NGpra5u56qbnab9s2bLFkGR88sknzVxp8/Okb2pqaoxnn33W6Nixo+Hv72/Ex8cbDz74oHHkyJEmq89iGE01dgMAAHBuMIcFAACYHoEFAACYHoEFAACYHoEFAACYHoEFAACYHoEFAACYHoEFAACYHoEFAACYHoEFAACYHoEFAACYHoEFAACYHoEFAACY3v8Hv2iWLd2zyfkAAAAASUVORK5CYII=\n"
          },
          "metadata": {}
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "The model has barely learned anything. What's going on? Can you improve on the performance e.g. by modifying the hyperparameters?"
      ],
      "metadata": {
        "id": "X74MuyuIjnLS"
      }
    }
  ],
  "metadata": {
    "colab": {
      "collapsed_sections": [
        "wo13ZXoZYB6J",
        "8KF9UtzUbrBA"
      ],
      "name": "hf_trainer_rnn.ipynb",
      "provenance": [],
      "include_colab_link": true
    },
    "kernelspec": {
      "display_name": "Python 3",
      "name": "python3"
    },
    "language_info": {
      "name": "python"
    },
    "widgets": {
      "application/vnd.jupyter.widget-state+json": {
        "601cc593e1c3455295499c1dd060cc84": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_445bff27d27e4852ae7b4124afd2e61d",
              "IPY_MODEL_dd9a38f5807d42f992fbed94e8d05670",
              "IPY_MODEL_35d2bc508fd04416a2c74f95704e6c19"
            ],
            "layout": "IPY_MODEL_2d06c2d7ec1d478cb173f5b2945e5893"
          }
        },
        "445bff27d27e4852ae7b4124afd2e61d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_527035e8d4474901a30e3e280f8678a5",
            "placeholder": "​",
            "style": "IPY_MODEL_8dbb4c21506d4e758fb47c4faf2057b6",
            "value": "Downloading readme: 100%"
          }
        },
        "dd9a38f5807d42f992fbed94e8d05670": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_c152ffd4292a401398ff2e9cdf29bc62",
            "max": 7809,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_0febb51b098144438b6864cef73596bc",
            "value": 7809
          }
        },
        "35d2bc508fd04416a2c74f95704e6c19": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_67d5b1f99c6d4ed399a582ec282eb481",
            "placeholder": "​",
            "style": "IPY_MODEL_b856be75aefc46bc881bc820330e9678",
            "value": " 7.81k/7.81k [00:00&lt;00:00, 16.3kB/s]"
          }
        },
        "2d06c2d7ec1d478cb173f5b2945e5893": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "527035e8d4474901a30e3e280f8678a5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8dbb4c21506d4e758fb47c4faf2057b6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "c152ffd4292a401398ff2e9cdf29bc62": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0febb51b098144438b6864cef73596bc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "67d5b1f99c6d4ed399a582ec282eb481": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b856be75aefc46bc881bc820330e9678": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "3f7a4384cdf94392a233e2ddaa2fbe8c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_17de7424a0914ba980c52fd1474fa3cc",
              "IPY_MODEL_29330125b3dc4c61a6a935cb4e999220",
              "IPY_MODEL_06c4cc10525c4282980a5092180790c0"
            ],
            "layout": "IPY_MODEL_83cd8fe2e6574af18f832c7df2708279"
          }
        },
        "17de7424a0914ba980c52fd1474fa3cc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3fed8e4d58744a8898636e4e99b94c09",
            "placeholder": "​",
            "style": "IPY_MODEL_18b35cc54dd945b5b216f8be5bf581b8",
            "value": "Downloading data: 100%"
          }
        },
        "29330125b3dc4c61a6a935cb4e999220": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e63fd4ccf25740939cc67a5479f89ec9",
            "max": 20979968,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_37ae6bc371164ddbac7492ca4ce6cc92",
            "value": 20979968
          }
        },
        "06c4cc10525c4282980a5092180790c0": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_132d7d5b41d74c708c52cf8ef66c199b",
            "placeholder": "​",
            "style": "IPY_MODEL_f7400c26a9c04b6ebda65d72b82ba4b7",
            "value": " 21.0M/21.0M [00:00&lt;00:00, 34.2MB/s]"
          }
        },
        "83cd8fe2e6574af18f832c7df2708279": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3fed8e4d58744a8898636e4e99b94c09": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "18b35cc54dd945b5b216f8be5bf581b8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e63fd4ccf25740939cc67a5479f89ec9": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "37ae6bc371164ddbac7492ca4ce6cc92": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "132d7d5b41d74c708c52cf8ef66c199b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f7400c26a9c04b6ebda65d72b82ba4b7": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e7a05a28271f4507afe255597b845c77": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_ae0e7e708e8047d7a2b3062d4e503a2c",
              "IPY_MODEL_c51bc41feca04969906bf87f556a61f6",
              "IPY_MODEL_33531cd5b4b24ad59c6c8747957a9a5d"
            ],
            "layout": "IPY_MODEL_3dc245576759495aa9605bbe78d09359"
          }
        },
        "ae0e7e708e8047d7a2b3062d4e503a2c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_6d18012122124723add21fed1dc1b83f",
            "placeholder": "​",
            "style": "IPY_MODEL_d3bb189c23da4f56aff2593e9a430a2f",
            "value": "Downloading data: 100%"
          }
        },
        "c51bc41feca04969906bf87f556a61f6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1ddd531895e248c8ab6d1338fc246913",
            "max": 20470363,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_9e5470e390b34b1ca8516d73ae42a773",
            "value": 20470363
          }
        },
        "33531cd5b4b24ad59c6c8747957a9a5d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3f3df2b09c694dfba1b7d2eb96a0cb0f",
            "placeholder": "​",
            "style": "IPY_MODEL_9f52edc6bdc44dd7993dc704a0e8bc83",
            "value": " 20.5M/20.5M [00:00&lt;00:00, 49.8MB/s]"
          }
        },
        "3dc245576759495aa9605bbe78d09359": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "6d18012122124723add21fed1dc1b83f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "d3bb189c23da4f56aff2593e9a430a2f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1ddd531895e248c8ab6d1338fc246913": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9e5470e390b34b1ca8516d73ae42a773": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "3f3df2b09c694dfba1b7d2eb96a0cb0f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9f52edc6bdc44dd7993dc704a0e8bc83": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "cb54f9f6f972425b89fb6ee19a63b1ba": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_6cbbab0ec97d4cf29f3299f4d86d942a",
              "IPY_MODEL_30b52d610cab4090a7ea704f2d94a7a8",
              "IPY_MODEL_cb32d657f1d54d2fa71ce28655d50ea9"
            ],
            "layout": "IPY_MODEL_a4b7bdc66e234a37bfe85bb76266bdd2"
          }
        },
        "6cbbab0ec97d4cf29f3299f4d86d942a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_41f7feb9fae040cba47e0974a8264ec4",
            "placeholder": "​",
            "style": "IPY_MODEL_ebe2001655194662b20ee5c1c31a1f63",
            "value": "Downloading data: 100%"
          }
        },
        "30b52d610cab4090a7ea704f2d94a7a8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_559a05a264ec478299b08884847d9f47",
            "max": 41996509,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_4cefa0cfde054ba0b6e08fcd2b86fb62",
            "value": 41996509
          }
        },
        "cb32d657f1d54d2fa71ce28655d50ea9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_10845084974c47f0bc80c5d9fd5e39eb",
            "placeholder": "​",
            "style": "IPY_MODEL_034010e0a32f4a6fab9638ca65eaa69e",
            "value": " 42.0M/42.0M [00:00&lt;00:00, 77.0MB/s]"
          }
        },
        "a4b7bdc66e234a37bfe85bb76266bdd2": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "41f7feb9fae040cba47e0974a8264ec4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ebe2001655194662b20ee5c1c31a1f63": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "559a05a264ec478299b08884847d9f47": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4cefa0cfde054ba0b6e08fcd2b86fb62": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "10845084974c47f0bc80c5d9fd5e39eb": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "034010e0a32f4a6fab9638ca65eaa69e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "beaaaf5eea884de690056f7bc0db35ad": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d78ddbb4a9f44931804dc2404072207a",
              "IPY_MODEL_63b41c445ccc4369af8b3c62d10caa27",
              "IPY_MODEL_a6be26f9d0b34cf8a3943e473afe6286"
            ],
            "layout": "IPY_MODEL_c68081a71c5e41eca1c0185ea6b234b5"
          }
        },
        "d78ddbb4a9f44931804dc2404072207a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_361a55924ed046238b5022b7b285f1b1",
            "placeholder": "​",
            "style": "IPY_MODEL_be1be61b1deb4cdb888f19df982f70a6",
            "value": "Generating train split: 100%"
          }
        },
        "63b41c445ccc4369af8b3c62d10caa27": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4ed0dafa0f894de1bab254e5781f7085",
            "max": 25000,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_967a4a82f8bb4014ac8a65712670db4c",
            "value": 25000
          }
        },
        "a6be26f9d0b34cf8a3943e473afe6286": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_ea4d3cbe6dd247f6bce1b9032f03e47d",
            "placeholder": "​",
            "style": "IPY_MODEL_997ade0ebda24450afc0fcd37d319581",
            "value": " 25000/25000 [00:00&lt;00:00, 52040.71 examples/s]"
          }
        },
        "c68081a71c5e41eca1c0185ea6b234b5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "361a55924ed046238b5022b7b285f1b1": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "be1be61b1deb4cdb888f19df982f70a6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "4ed0dafa0f894de1bab254e5781f7085": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "967a4a82f8bb4014ac8a65712670db4c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "ea4d3cbe6dd247f6bce1b9032f03e47d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "997ade0ebda24450afc0fcd37d319581": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "f69c549312234802b6ed88b1469075e3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_3bb78037136d49b1b0d66718b63b359a",
              "IPY_MODEL_957c896524b9431393e1dca0d9fdcabb",
              "IPY_MODEL_ed4ea35ff9394cb3a29a50a41848270c"
            ],
            "layout": "IPY_MODEL_545b739cc7eb47e39f637e84a516ad3b"
          }
        },
        "3bb78037136d49b1b0d66718b63b359a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_367a65e921064433b1d85a05c2c364aa",
            "placeholder": "​",
            "style": "IPY_MODEL_8b79020824864c7b9a9374cd75986b89",
            "value": "Generating test split: 100%"
          }
        },
        "957c896524b9431393e1dca0d9fdcabb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_e68e65143753434ca61778c1342356d7",
            "max": 25000,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_b717d2a5bc8a49459c9147a17d8794dc",
            "value": 25000
          }
        },
        "ed4ea35ff9394cb3a29a50a41848270c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a52638aafb934b60b982203c4061e24e",
            "placeholder": "​",
            "style": "IPY_MODEL_8b2d730dc0e14ef680de56a648d13aeb",
            "value": " 25000/25000 [00:00&lt;00:00, 54077.61 examples/s]"
          }
        },
        "545b739cc7eb47e39f637e84a516ad3b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "367a65e921064433b1d85a05c2c364aa": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8b79020824864c7b9a9374cd75986b89": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "e68e65143753434ca61778c1342356d7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b717d2a5bc8a49459c9147a17d8794dc": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a52638aafb934b60b982203c4061e24e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8b2d730dc0e14ef680de56a648d13aeb": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d18d65a927494eb5a567abdeb71a7f93": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_acbe022ef92042d08e523e66571a68a1",
              "IPY_MODEL_1d8a01207976474e8b5b2e8189839929",
              "IPY_MODEL_b812780654c743c5b6ea1783a80b0929"
            ],
            "layout": "IPY_MODEL_a5d11c40fb294ec8bebe8742c9991022"
          }
        },
        "acbe022ef92042d08e523e66571a68a1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5f866679aaa042c89358c609533c9ade",
            "placeholder": "​",
            "style": "IPY_MODEL_25beb6fb3dda462fb308d030f0051700",
            "value": "Generating unsupervised split: 100%"
          }
        },
        "1d8a01207976474e8b5b2e8189839929": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_5b8b1e48d0154d5587d5b0dd92307309",
            "max": 50000,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_246a45f1e8b24b369a0b27ba035fe5ff",
            "value": 50000
          }
        },
        "b812780654c743c5b6ea1783a80b0929": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_38b640e024b24fd79349d40699ddc95d",
            "placeholder": "​",
            "style": "IPY_MODEL_bd38d55fd0714892aeefd06f08e91303",
            "value": " 50000/50000 [00:01&lt;00:00, 47342.33 examples/s]"
          }
        },
        "a5d11c40fb294ec8bebe8742c9991022": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5f866679aaa042c89358c609533c9ade": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "25beb6fb3dda462fb308d030f0051700": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "5b8b1e48d0154d5587d5b0dd92307309": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "246a45f1e8b24b369a0b27ba035fe5ff": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "38b640e024b24fd79349d40699ddc95d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bd38d55fd0714892aeefd06f08e91303": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "1a2e80a6758447719a574f0b30741487": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_d09a31f1abc949f1b5630c2b225750f9",
              "IPY_MODEL_c288a59070c048f69a20e59ae49947a9",
              "IPY_MODEL_f8d20844505046368fbb6bff2ac00cb8"
            ],
            "layout": "IPY_MODEL_c86cdac9336d468aa74f69642244482b"
          }
        },
        "d09a31f1abc949f1b5630c2b225750f9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_240339f70ee6435bbd5efc6accf66863",
            "placeholder": "​",
            "style": "IPY_MODEL_0ac4f9a91a78497389cbe3002d3f0924",
            "value": "tokenizer_config.json: 100%"
          }
        },
        "c288a59070c048f69a20e59ae49947a9": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7b1ebe4cf37940b5b7666f653b9d88df",
            "max": 49,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_e53ab6a12d774a66aa8ac5da4b96f714",
            "value": 49
          }
        },
        "f8d20844505046368fbb6bff2ac00cb8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_72382cc803174b3da35596eee5e8a3cd",
            "placeholder": "​",
            "style": "IPY_MODEL_dd9640a5488e44a69b94d2d5770f4f2e",
            "value": " 49.0/49.0 [00:00&lt;00:00, 761B/s]"
          }
        },
        "c86cdac9336d468aa74f69642244482b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "240339f70ee6435bbd5efc6accf66863": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0ac4f9a91a78497389cbe3002d3f0924": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "7b1ebe4cf37940b5b7666f653b9d88df": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "e53ab6a12d774a66aa8ac5da4b96f714": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "72382cc803174b3da35596eee5e8a3cd": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dd9640a5488e44a69b94d2d5770f4f2e": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "0f70760177464f55b77db49629f8b4be": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_fc96b4b213064fd5b8f096b4ff117357",
              "IPY_MODEL_97173b8da10e4312b7b419d448fe27a3",
              "IPY_MODEL_4003709e9752457ea2ad3e5aeb86543f"
            ],
            "layout": "IPY_MODEL_ca4c31ccdc6d4d2a96bc5bfe3bdd6c70"
          }
        },
        "fc96b4b213064fd5b8f096b4ff117357": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9459564b479149cab3e45a05ed52360e",
            "placeholder": "​",
            "style": "IPY_MODEL_0afed6b2c1024f199cacfdedb8cedd03",
            "value": "config.json: 100%"
          }
        },
        "97173b8da10e4312b7b419d448fe27a3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_b04bf0fbca71412595b03d96d6bd210d",
            "max": 570,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_f67a7ac769cf43a18fbb0445654101ff",
            "value": 570
          }
        },
        "4003709e9752457ea2ad3e5aeb86543f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_33b11d1b526142f9a3a45e198847a33e",
            "placeholder": "​",
            "style": "IPY_MODEL_99c7b885e0e949b584d27e9c866faf06",
            "value": " 570/570 [00:00&lt;00:00, 13.4kB/s]"
          }
        },
        "ca4c31ccdc6d4d2a96bc5bfe3bdd6c70": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "9459564b479149cab3e45a05ed52360e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "0afed6b2c1024f199cacfdedb8cedd03": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "b04bf0fbca71412595b03d96d6bd210d": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "f67a7ac769cf43a18fbb0445654101ff": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "33b11d1b526142f9a3a45e198847a33e": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "99c7b885e0e949b584d27e9c866faf06": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9d27c45cffad47cdb1a5788e245b8aef": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_7b98864f4e824a6db2022a25a7597c2c",
              "IPY_MODEL_a428356220f94d069f2129de8a4824b8",
              "IPY_MODEL_d3bc3fa81af5424482b3a797ad198797"
            ],
            "layout": "IPY_MODEL_da7e4f5bb7854eaea4c7e9594ca03399"
          }
        },
        "7b98864f4e824a6db2022a25a7597c2c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_44b1a777f5794986a17fe474578b1373",
            "placeholder": "​",
            "style": "IPY_MODEL_b8ce620748fd43f988948711220518b1",
            "value": "vocab.txt: 100%"
          }
        },
        "a428356220f94d069f2129de8a4824b8": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9595357778854e0eb6f6468737e2b396",
            "max": 213450,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_dc9d236e4f5c45428285bb9eb3668598",
            "value": 213450
          }
        },
        "d3bc3fa81af5424482b3a797ad198797": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_a63d3bdc7830487bac7fd5c11e0418d4",
            "placeholder": "​",
            "style": "IPY_MODEL_47ec1edee0d34965944cf295c0aa1d13",
            "value": " 213k/213k [00:00&lt;00:00, 1.63MB/s]"
          }
        },
        "da7e4f5bb7854eaea4c7e9594ca03399": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "44b1a777f5794986a17fe474578b1373": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "b8ce620748fd43f988948711220518b1": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9595357778854e0eb6f6468737e2b396": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "dc9d236e4f5c45428285bb9eb3668598": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "a63d3bdc7830487bac7fd5c11e0418d4": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "47ec1edee0d34965944cf295c0aa1d13": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d45fb6b1db644a6f90626069345b452f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_e23a7f66f25e43069e58662f37e1a397",
              "IPY_MODEL_84abe125a60542c0a5f664bd3e891172",
              "IPY_MODEL_84186a90796049c998ba69b8dc3b1004"
            ],
            "layout": "IPY_MODEL_4eeb94d9f3c64808b4b82feecdd1fe26"
          }
        },
        "e23a7f66f25e43069e58662f37e1a397": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_8a54a001c2124c0694e4981350685da5",
            "placeholder": "​",
            "style": "IPY_MODEL_eb5fef2ec0564d0782025d29db163e8f",
            "value": "tokenizer.json: 100%"
          }
        },
        "84abe125a60542c0a5f664bd3e891172": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_d0dda846da934384a2ee06bc140543bf",
            "max": 435797,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_00b4f767f0e64004be48768f7bc34fd3",
            "value": 435797
          }
        },
        "84186a90796049c998ba69b8dc3b1004": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_1210f69cbb68400e8474ba10046be85b",
            "placeholder": "​",
            "style": "IPY_MODEL_07c4b00ae6aa45c9b7e3036e66bd5588",
            "value": " 436k/436k [00:00&lt;00:00, 6.09MB/s]"
          }
        },
        "4eeb94d9f3c64808b4b82feecdd1fe26": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8a54a001c2124c0694e4981350685da5": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "eb5fef2ec0564d0782025d29db163e8f": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "d0dda846da934384a2ee06bc140543bf": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "00b4f767f0e64004be48768f7bc34fd3": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "1210f69cbb68400e8474ba10046be85b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "07c4b00ae6aa45c9b7e3036e66bd5588": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "834bf849fbbb421fb69f848a1e4f4080": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_102a88efa1ea4693a096482cacd54727",
              "IPY_MODEL_02bc83cc4d0c4a6facad8ebab1349d01",
              "IPY_MODEL_87d0828eac384e459355cc292eacd519"
            ],
            "layout": "IPY_MODEL_73ec6b13c83744b99ce7bbbb5c01385f"
          }
        },
        "102a88efa1ea4693a096482cacd54727": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_af55a9f236c94b4fadef6a02829a387f",
            "placeholder": "​",
            "style": "IPY_MODEL_11743d1ce51245708e60dc6e04bfef71",
            "value": "Map: 100%"
          }
        },
        "02bc83cc4d0c4a6facad8ebab1349d01": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_508d1dd5624048cb90a75e7f8b088037",
            "max": 25000,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_560039c590614a3da97e3ee1384106c4",
            "value": 25000
          }
        },
        "87d0828eac384e459355cc292eacd519": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_2316c6176a5d40658a3afa8962914e51",
            "placeholder": "​",
            "style": "IPY_MODEL_ddb780f98876493eaf26d7ed344e4330",
            "value": " 25000/25000 [01:21&lt;00:00, 591.92 examples/s]"
          }
        },
        "73ec6b13c83744b99ce7bbbb5c01385f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "af55a9f236c94b4fadef6a02829a387f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "11743d1ce51245708e60dc6e04bfef71": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "508d1dd5624048cb90a75e7f8b088037": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "560039c590614a3da97e3ee1384106c4": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "2316c6176a5d40658a3afa8962914e51": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "ddb780f98876493eaf26d7ed344e4330": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "91a865277aa4458eb6e872895f698f41": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_38f87798046f4ee68f301e51cf329cb6",
              "IPY_MODEL_cccc86f8fc504edd948d683b07a685c2",
              "IPY_MODEL_f0977ede73fe4bf3848ac3bf3f1b6403"
            ],
            "layout": "IPY_MODEL_ce5602958e5a45ca91076294c73aaafb"
          }
        },
        "38f87798046f4ee68f301e51cf329cb6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_3d38c3b8932b46088c3b4d762c3291a7",
            "placeholder": "​",
            "style": "IPY_MODEL_193c6066997549e7b48c8d9842b352b6",
            "value": "Map: 100%"
          }
        },
        "cccc86f8fc504edd948d683b07a685c2": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_546e890771044b1d9f245856ca199f7b",
            "max": 25000,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_4b8cb267340248d281eccf3d75f02738",
            "value": 25000
          }
        },
        "f0977ede73fe4bf3848ac3bf3f1b6403": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_4f7dd067bf234ee0b136d7c0245710a8",
            "placeholder": "​",
            "style": "IPY_MODEL_5518613d33944dc7a50f1fc0c8797109",
            "value": " 25000/25000 [00:31&lt;00:00, 933.62 examples/s]"
          }
        },
        "ce5602958e5a45ca91076294c73aaafb": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "3d38c3b8932b46088c3b4d762c3291a7": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "193c6066997549e7b48c8d9842b352b6": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "546e890771044b1d9f245856ca199f7b": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "4b8cb267340248d281eccf3d75f02738": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "4f7dd067bf234ee0b136d7c0245710a8": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "5518613d33944dc7a50f1fc0c8797109": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "154569f9633046af8a4bae96088b832c": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HBoxModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HBoxModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HBoxView",
            "box_style": "",
            "children": [
              "IPY_MODEL_63aede7845044c29ad2ec0436339a177",
              "IPY_MODEL_7a0bc7df773048ce91b1697044a84118",
              "IPY_MODEL_81ecb2b2d8d340f98ad5c363a758e42a"
            ],
            "layout": "IPY_MODEL_7b3b0424b1a449aa82010f96c16d213a"
          }
        },
        "63aede7845044c29ad2ec0436339a177": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_7ab3d996813446398b7de30bb3c88f36",
            "placeholder": "​",
            "style": "IPY_MODEL_8de4b5f9b2374eee99a99a110480401d",
            "value": "Downloading builder script: 100%"
          }
        },
        "7a0bc7df773048ce91b1697044a84118": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "FloatProgressModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "FloatProgressModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "ProgressView",
            "bar_style": "success",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9fb2dabeecee4172985b38f93163053c",
            "max": 4203,
            "min": 0,
            "orientation": "horizontal",
            "style": "IPY_MODEL_cbb4fe0681a84febbe5109d237097b06",
            "value": 4203
          }
        },
        "81ecb2b2d8d340f98ad5c363a758e42a": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "HTMLModel",
          "model_module_version": "1.5.0",
          "state": {
            "_dom_classes": [],
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "HTMLModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/controls",
            "_view_module_version": "1.5.0",
            "_view_name": "HTMLView",
            "description": "",
            "description_tooltip": null,
            "layout": "IPY_MODEL_9207061018444358875b76f16560dc8f",
            "placeholder": "​",
            "style": "IPY_MODEL_bd71b5eea5414cf69a85b58dcd8689ac",
            "value": " 4.20k/4.20k [00:00&lt;00:00, 258kB/s]"
          }
        },
        "7b3b0424b1a449aa82010f96c16d213a": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "7ab3d996813446398b7de30bb3c88f36": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "8de4b5f9b2374eee99a99a110480401d": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        },
        "9fb2dabeecee4172985b38f93163053c": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "cbb4fe0681a84febbe5109d237097b06": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "ProgressStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "ProgressStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "bar_color": null,
            "description_width": ""
          }
        },
        "9207061018444358875b76f16560dc8f": {
          "model_module": "@jupyter-widgets/base",
          "model_name": "LayoutModel",
          "model_module_version": "1.2.0",
          "state": {
            "_model_module": "@jupyter-widgets/base",
            "_model_module_version": "1.2.0",
            "_model_name": "LayoutModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "LayoutView",
            "align_content": null,
            "align_items": null,
            "align_self": null,
            "border": null,
            "bottom": null,
            "display": null,
            "flex": null,
            "flex_flow": null,
            "grid_area": null,
            "grid_auto_columns": null,
            "grid_auto_flow": null,
            "grid_auto_rows": null,
            "grid_column": null,
            "grid_gap": null,
            "grid_row": null,
            "grid_template_areas": null,
            "grid_template_columns": null,
            "grid_template_rows": null,
            "height": null,
            "justify_content": null,
            "justify_items": null,
            "left": null,
            "margin": null,
            "max_height": null,
            "max_width": null,
            "min_height": null,
            "min_width": null,
            "object_fit": null,
            "object_position": null,
            "order": null,
            "overflow": null,
            "overflow_x": null,
            "overflow_y": null,
            "padding": null,
            "right": null,
            "top": null,
            "visibility": null,
            "width": null
          }
        },
        "bd71b5eea5414cf69a85b58dcd8689ac": {
          "model_module": "@jupyter-widgets/controls",
          "model_name": "DescriptionStyleModel",
          "model_module_version": "1.5.0",
          "state": {
            "_model_module": "@jupyter-widgets/controls",
            "_model_module_version": "1.5.0",
            "_model_name": "DescriptionStyleModel",
            "_view_count": null,
            "_view_module": "@jupyter-widgets/base",
            "_view_module_version": "1.2.0",
            "_view_name": "StyleView",
            "description_width": ""
          }
        }
      }
    },
    "accelerator": "GPU"
  },
  "nbformat": 4,
  "nbformat_minor": 0
}