id_NNSplit_trainer.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "5HlpOyhRdgm4"
   },
   "source": [
    "This notebook shows how to train [NNSplit](https://github.com/bminixhofer/nnsplit/) on a custom dataset and load it for inference."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "PF8h9mSzRnCJ"
   },
   "source": [
    "# Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "XYvCFh9cdrZ3"
   },
   "source": [
    "First, clone the Github Repo and install requirements. If you are running this on Colab, you will likely have to restart the runtime after installing the requirements because of some version mismatches."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 5715,
     "status": "ok",
     "timestamp": 1681027678525,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "smd7hIM_zcLO",
    "outputId": "fb250abc-dc6e-42f0-ee19-59c2b70279f5",
    "tags": []
   },
   "outputs": [],
   "source": [
    "# !git clone https://www.github.com/bminixhofer/nnsplit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create or activate your venv so that it won't conflict with your base env\n",
    "\n",
    "Additional:\n",
    "If you don't have jupyter kernel or venv and want to activate one, please check the following link:\n",
    "https://janakiev.com/blog/jupyter-virtual-envs/"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# !bash prepare_nnsplit_env.sh"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8vFPPwbuQ9v0",
    "tags": []
   },
   "source": [
    "# Data preparation"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "N74zUHFLd2aK"
   },
   "source": [
    "Training NNSplit is not limited to a specific dataset. Howevever, I have found the [Linguatools Wikipedia Dumps](https://linguatools.org/tools/corpora/wikipedia-monolingual-corpora/) to work well, so there is built-in functionality to load those. Feel free to use other data!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "VGEolHUkeIHK"
   },
   "source": [
    "First, download the `.xml.bz2` file and unzip it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 36454,
     "status": "ok",
     "timestamp": 1680974642407,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "uabNv05OneLK",
    "outputId": "7fab65b3-717f-4019-956d-655102f6b55d"
   },
   "outputs": [],
   "source": [
    "!wget \"https://www.dropbox.com/s/cnrhd11zdtc1pic/idwiki-20181001-corpus.xml.bz2?dl=1\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 16,
     "status": "ok",
     "timestamp": 1680974642407,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "L4aHHmAEoj25"
   },
   "outputs": [],
   "source": [
    "!mv idwiki-20181001-corpus.xml.bz2?dl=1 idwiki-20181001-corpus.xml.bz2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 1119757,
     "status": "ok",
     "timestamp": 1680975762149,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "1GY4mHCboaUu"
   },
   "outputs": [],
   "source": [
    "!bzip2 -d idwiki-20181001-corpus.xml.bz2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "UArMGRYReVGT"
   },
   "source": [
    "Now we can create the dataset. `xml_dump_iter` is one of the built in methods which yields an iterator over all texts in the wikipedia dump, trying to remove tags and other markup."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 1500,
     "status": "ok",
     "timestamp": 1680975763634,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "74lGj78HszTo"
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.append(\"nnsplit/train\")\n",
    "from text_data import MemoryMapDataset, xml_dump_iter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 160
    },
    "executionInfo": {
     "elapsed": 8,
     "status": "ok",
     "timestamp": 1680975763634,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "tQ4Cedjws7e6",
    "outputId": "41b59753-07cb-4e66-d48e-7703ec884edd"
   },
   "outputs": [],
   "source": [
    "# xml_iter = xml_dump_iter(\"enwiki-20181001-corpus.xml\", \n",
    "xml_iter = xml_dump_iter(\"idwiki-20181001-corpus.xml\", \n",
    "                         min_text_length=300, \n",
    "                         max_text_length=5000)\n",
    "next(xml_iter)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "KMazX62Pe66B"
   },
   "source": [
    "`MemoryMapDataset` is another convient built-in class, but not specific to the Wikipedia dump. It is a `torch.utils.data.Dataset` which can be created using a `texts.txt` and `slices.pkl` file. The `texts.txt` file is [memory-mapped](https://en.wikipedia.org/wiki/Memory-mapped_file) and `slices.pkl` contains a Python array with indices that determine at which position in the dataset which range of the text should be loaded. This allows accessing each text without ever loading all the data into memory.\n",
    "\n",
    "To create `texts.txt` and `slices.pkl` from an iterator over text, use `MemoryMapDataset.iterator_to_text_and_slices`.\n",
    "\n",
    "Note that this will be quite slow since iterating over the XML dump takes a significant amount of time, so I would recommend caching `texts.txt` and `slices.pkl` somewhere.\n",
    "\n",
    "`max_n_texts=10_000_000` is only needed in Colab to keep disk usage in check, feel free to remove this otherwise."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 49,
     "referenced_widgets": [
      "9dc970e0ffa840e5978c294f3e851d3d",
      "1460af1ac54644edbaf21bcc5c197995",
      "2ca1fa79adeb47edbe57668b09c7bbed",
      "0968a28f667d4410ab4504a8432af229",
      "93dfab7bd9734a639f6f7622522b29fb",
      "56c206cb7f77436684fcbfc3c7ff1abf",
      "2deda6ac0ad243dbadc55c394b8f6baf",
      "ea186d2a6dbe405d98f6b48769c671d4",
      "189110f7cf5d4595b5648c6bdcdec2ab",
      "ab43e58f2ae54f9d9ad243a9a08afd7b",
      "a3c26d6b7d204e67b1f0cfb2291a0786"
     ]
    },
    "executionInfo": {
     "elapsed": 2911495,
     "status": "ok",
     "timestamp": 1680978675122,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "zlRunloks4RU",
    "outputId": "1d6748f2-7c8c-4e4e-91e6-37e5994ae99a"
   },
   "outputs": [],
   "source": [
    "# xml_iter = xml_dump_iter(\"enwiki-20181001-corpus.xml\", \n",
    "xml_iter = xml_dump_iter(\"idwiki-20181001-corpus.xml\", \n",
    "                         min_text_length=300,\n",
    "                         max_text_length=5000)\n",
    "MemoryMapDataset.iterator_to_text_and_slices(xml_iter, \n",
    "                                             \"texts.txt\", \n",
    "                                             \"slices.pkl\",\n",
    "                                             max_n_texts=10_000_000)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "BGRK18nRKLb9",
    "tags": []
   },
   "source": [
    "# Data Load/Dump"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "y1BvQ894gvKJ"
   },
   "source": [
    "Here, I am saving the outputs to my Drive, you will have to adjust these paths."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 1132727,
     "status": "ok",
     "timestamp": 1681028977242,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "Q8_31wGhajDR",
    "outputId": "6dc6f9da-4b8c-4650-bb22-f513658c4c7a"
   },
   "outputs": [],
   "source": [
    "# #mount gdrive\n",
    "# from google.colab import drive\n",
    "# drive.mount('/content/drive')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 74435,
     "status": "ok",
     "timestamp": 1681029051671,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "xQs7qQHHtXVH"
   },
   "outputs": [],
   "source": [
    "#if true, export. if false, import\n",
    "export = False\n",
    "lang = \"id\"\n",
    "\n",
    "if export:\n",
    "  !mkdir \"../NNSplit_lib\"\n",
    "  !mv slices.pkl \"../NNSplit_lib/slices_{lang}.pkl\"\n",
    "  !mv texts.txt \"../NNSplit_lib/texts_{lang}.txt\"\n",
    "else:\n",
    "  !cp -a \"../NNSplit_lib/slices_{lang}.pkl\" slices.pkl\n",
    "  !cp -a \"../NNSplit_lib/texts_{lang}.txt\" texts.txt"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "AZpLGc_kPlOq"
   },
   "source": [
    "# Training"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Nh9UKxN4hFuw"
   },
   "source": [
    "Now we can get started with training!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 4,
     "status": "ok",
     "timestamp": 1681029051672,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "Ju0AhUATnvRx"
   },
   "outputs": [],
   "source": [
    "import sys\n",
    "sys.path.append(\"train\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 4008,
     "status": "ok",
     "timestamp": 1681029055677,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "X96MCCfQRBDG"
   },
   "outputs": [],
   "source": [
    "import json\n",
    "from pytorch_lightning.trainer import Trainer\n",
    "from tqdm.auto import tqdm\n",
    "from model import Network\n",
    "from text_data import MemoryMapDataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "ogqWNcZkhKEB"
   },
   "source": [
    "NNSplit has a `Network` class which is a `pl.LightningModule` specifying network architecture, data loading logic etc. To instantiate a new network, we need to first get the default hyperparameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 397,
     "status": "ok",
     "timestamp": 1681029472121,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "mKNM1sVpR-Lq",
    "outputId": "4fefaef9-c6d9-401e-c658-1745d0ce11cb"
   },
   "outputs": [],
   "source": [
    "parser = Network.get_parser()\n",
    "hparams = parser.parse_args([])\n",
    "hparams"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "vMldDgFkS3ZU"
   },
   "source": [
    "## Load text data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "QE2YMQOrhy24"
   },
   "source": [
    "Next, we can load the text data created previously."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 2417,
     "status": "ok",
     "timestamp": 1681029058081,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "ADW2U60kSnOf"
   },
   "outputs": [],
   "source": [
    "text_dataset = MemoryMapDataset(\"texts.txt\", \"slices.pkl\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "8YOPVi2vh2dG"
   },
   "source": [
    "Keep in mind that this can be any `torch.utils.data.Dataset` with `str` entries, so you can completely customize it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 178
    },
    "executionInfo": {
     "elapsed": 9,
     "status": "ok",
     "timestamp": 1681029058081,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "hS-KFyiOS8Bj",
    "outputId": "1004dc0b-da30-4737-ee85-f29f3cb0e9d9"
   },
   "outputs": [],
   "source": [
    "text_dataset[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "GEDFRxU_S-4w"
   },
   "source": [
    "## Load labeler"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "I5cVdSTFiE9o"
   },
   "source": [
    "Next, create a `Labeler`, which is used to annotate the text from above. Any SpaCy model which supports sentencization can be used. You will have to install the appropriate SpaCy model with `python -m spacy ...` when running this in Colab."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 7332,
     "status": "ok",
     "timestamp": 1681029065407,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "tkwjwBPiCUPl"
   },
   "outputs": [],
   "source": [
    "from labeler import Labeler, SpacySentenceTokenizer, SpacyWordTokenizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import en_core_web_sm\n",
    "nlp = en_core_web_sm.load()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 1005,
     "status": "ok",
     "timestamp": 1681029066395,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "q568uRViSHco"
   },
   "outputs": [],
   "source": [
    "labeler = Labeler(\n",
    "    [\n",
    "        SpacySentenceTokenizer(\n",
    "            \"en_core_web_sm\", lower_start_prob=0.7, remove_end_punct_prob=0.7, punctuation=\".\"\n",
    "        ),\n",
    "        SpacyWordTokenizer(\"en_core_web_sm\"),\n",
    "    ]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "aps02am6iio_"
   },
   "source": [
    "`Labeler.visualize` shows you what the network sees: \n",
    "- `byte` is the UTF-8 encoded text. This has changed in the newest version of NNSplit. Previously characters where used, but using bytes allows NNSplit to work for any language regardless of the characters used to represent it.\n",
    "- The other rows depend on the `Labeler` and determine what the neural networks tries to predict."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 7,
     "status": "ok",
     "timestamp": 1681029066395,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "IL5-_awsSfYX",
    "outputId": "a9aa66b8-deba-43c9-a84b-b54e96c10be5"
   },
   "outputs": [],
   "source": [
    "labeler.visualize(\"This is a test. This is another test.\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CL7gZKuITDYA"
   },
   "source": [
    "## Start training!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "6_cI7PJ0jNfH"
   },
   "source": [
    "Now we can finally start training. \n",
    "\n",
    "`train_size` determines how many entries in the dataset to sample for each epoch. \n",
    "\n",
    "Using SpaCy with multiprocessing leaks memory, so the memory usage will continously increase during each epoch and reset at the end. So you will have to set `train_size` to a size that corresponds to how much memory is available. `500_000` works well in Colab.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "os.environ['CUDA_LAUNCH_BLOCKING'] = \"1\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 457,
     "status": "ok",
     "timestamp": 1681029477550,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "3D14-MKkTUL3"
   },
   "outputs": [],
   "source": [
    "hparams.gpus = 1\n",
    "hparams.max_epochs = 4\n",
    "hparams.train_size = 100000\n",
    "hparams.predict_indices = [0, 1] # which split levels of the labeler to predict\n",
    "# how to weigh the selected indices\n",
    "# in general sentence boundary detection should be weighed the highest\n",
    "hparams.level_weights = [0.1, 2.0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fq2u1s6Cj5sT"
   },
   "source": [
    "Instantiate the network."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 4,
     "status": "ok",
     "timestamp": 1681029440140,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "GV385cJtTNVN",
    "outputId": "bae5ac11-6d17-40fb-c176-44bf806e0316"
   },
   "outputs": [],
   "source": [
    "# https://discuss.pytorch.org/t/pytorch-lightning-module-cant-set-attribute-error/121125/5\n",
    "model = Network(\n",
    "  text_dataset,\n",
    "  labeler,\n",
    "  hparams,\n",
    ")\n",
    "model"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "OHDW5wbKj8xz"
   },
   "source": [
    "Instantiate the `pl.trainer.Trainer`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/"
    },
    "executionInfo": {
     "elapsed": 347,
     "status": "ok",
     "timestamp": 1681029489986,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "TzV01atSTjKe",
    "outputId": "0832fb63-15c0-4b6a-8367-aaa39e982e74"
   },
   "outputs": [],
   "source": [
    "trainer = Trainer.from_argparse_args(hparams)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "niq9NiB6Kp9V"
   },
   "source": [
    "Remove variables that has been inserted to Network class to save RAM"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 204
    },
    "executionInfo": {
     "elapsed": 534,
     "status": "error",
     "timestamp": 1681029483853,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "Jaasfh3BKb_W",
    "outputId": "833742f2-547b-4494-d5f6-83e24d3837f2"
   },
   "outputs": [],
   "source": [
    "del text_dataset, labeler, hparams"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "o6_n-DuNkD0o"
   },
   "source": [
    "And fit the model. Each row of the f1 and precision scores corresponds to each tokenizer of the `Labeler`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "colab": {
     "base_uri": "https://localhost:8080/",
     "height": 1000,
     "referenced_widgets": [
      "60f1b29c27ff425c919327eb608a16c6",
      "8f27cc99c5354e4f95330747277cc318",
      "d15b9226cd384d8096aadccb8c948ae3",
      "82044626fe8c4295b0e01a4145b8217b",
      "34667acf38a648a68f8af315b855ab18",
      "23374edc4a994c63a0abf9428dd41838",
      "3f7964262d2949f5ac0d45f640b1dfd1",
      "397213b253854f2a9471401bcc51d4b4",
      "9edde47996ef46c0b2739468d078746d",
      "df17aed724bb4055968643aa9105aea9",
      "ec63970793e945ecb1404930d6c6aaac",
      "dd749566875040c8890c6e270f8c3c34",
      "db79ae604d284efb9984ec1004d1a460",
      "920f552635234c1f9f46c39f91c3ab2f",
      "83e0dbc5b6d44a729ac5d1180d037252",
      "b26c08b8114a4b8a96e981be7d6e46ca",
      "60b5c2ee79c74150821be95c53fc43b2",
      "0819be7a9cd8434484b6c7e62c33f6b5",
      "de35f8ce97194099a185915e3856dd1b",
      "b6269e82c5a34514aeadc3f36ca7fd10",
      "24661beeac614ce9873b3a8adbc8d428",
      "87fb86503e1449dba9965267bf862425",
      "ae03952b50a6425682d72f1b82669ebc",
      "5206d17ff7f14de6b8e313b59ee28f3b",
      "52ea02224dee45c1b355be493c84b54f",
      "0267c450243a42d08c5c0a6b6ebb3c43",
      "d1f339779e5943deaed85a3b9e6dd5cf",
      "39ab6d5a11d4495e8cfeab49bf65fa9a",
      "74a818e188604a9cbea85c43b0844d10",
      "103f112320b844498c1946cd1564bfee",
      "84642daaa6bd4f449ca05c0b51086ebf",
      "348d359fec60427285df8e21a26c54a1",
      "3e5ae13a018a471da030f3bfefa3337d"
     ]
    },
    "id": "Awa3ZO5oTl7X",
    "outputId": "97bdedff-242c-4cec-f1db-646704deb563"
   },
   "outputs": [],
   "source": [
    "trainer.fit(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Iq16P_pqklch"
   },
   "source": [
    "Finally, store the trained model somewhere. This saves a `.onnx` export of the model in the specified directory."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 3,
     "status": "aborted",
     "timestamp": 1680986837482,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "s5tSj4DGoYfL"
   },
   "outputs": [],
   "source": [
    "# onnx metadata which determines how to use the prediction indices to split text\n",
    "metadata = {\n",
    "    \"split_sequence\": json.dumps(\n",
    "        {\n",
    "            \"instructions\": [\n",
    "                [\"Sentence\", {\"PredictionIndex\": 0}],\n",
    "                [\"Token\", {\"PredictionIndex\": 1}],\n",
    "                [\"_Whitespace\", {\"Function\": \"whitespace\"}],\n",
    "            ]\n",
    "        }\n",
    "    )\n",
    "}\n",
    "model.store(\"id\", metadata)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "executionInfo": {
     "elapsed": 4,
     "status": "aborted",
     "timestamp": 1680986837483,
     "user": {
      "displayName": "Maulana Akbar",
      "userId": "02353462897995776446"
     },
     "user_tz": -420
    },
    "id": "bCYPGhO0UqXV",
    "tags": []
   },
   "outputs": [],
   "source": [
    "!mkdir \"../NNSplit_lib/nnsplit_id_model\"\n",
    "!cp -a -r id/* \"../NNSplit_lib/nnsplit_id_model\""
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "tWGZJRWAl6w-",
    "tags": []
   },
   "source": [
    "# Can be skipped"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Ynlr1bYuk_ZV"
   },
   "source": [
    "First, install NNSplit."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "2y8mTjA9YRXH"
   },
   "outputs": [],
   "source": [
    "from nnsplit import NNSplit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "a7tfpxtplE8S"
   },
   "source": [
    "Instantiate the splitter."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fiAxbt37YFYP",
    "tags": []
   },
   "source": [
    "# Load the custom model in NNSplit"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "3EeDhEjoYj-a"
   },
   "outputs": [],
   "source": [
    "splitter = NNSplit(\"en/model.onnx\", use_cuda=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "zrjzXPnPlPDs"
   },
   "source": [
    "And split a text!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "ERhaXDUlYmcU"
   },
   "outputs": [],
   "source": [
    "splits = splitter.split([\"This is a test This is another test.\"])[0]\n",
    "splits"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "WOi2xVTllSjv"
   },
   "source": [
    "The public API of NNSplit has changed significantly, making it much easier to use now. Everything is a `nnsplit.Split` which can be iterated over or stringified with `str(...)`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "L5DRnwAJY5Px"
   },
   "outputs": [],
   "source": [
    "for sentence in splits:\n",
    "    print(str(sentence).ljust(30), type(sentence))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "yRt37H-hlmqf"
   },
   "source": [
    "Or if you want to go token-level:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "7mKlF78WY-31"
   },
   "outputs": [],
   "source": [
    "for sentence in splits:\n",
    "    for token in sentence:\n",
    "        print(str(token).ljust(10), repr(token).ljust(30), type(token))\n",
    "\n",
    "    print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "YyVn5FHdluOQ"
   },
   "source": [
    "Until the smallest unit, which then returns a `str` instead of an `nnsplit.Split`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "xTgFJU1uZgnM"
   },
   "outputs": [],
   "source": [
    "for sentence in splits:\n",
    "    for [text, whitespace] in sentence:\n",
    "        print(text.ljust(10), type(text))\n",
    "        print(f'\"{whitespace}\"'.ljust(10), type(whitespace))\n",
    "        print()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "wrN4T1lYl3is"
   },
   "source": [
    "Finally, for some benchmarks: If you are running `NNSplit` on GPU, you can increase the speed on large datasets by using a big batch size."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "uYHT8F79bP89"
   },
   "outputs": [],
   "source": [
    "splitter = NNSplit(\"en/model.onnx\", use_cuda=True, batch_size=2**14)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "0BSrcu55bDQV"
   },
   "outputs": [],
   "source": [
    "text = \"This is a test This is another test.\"\n",
    "\n",
    "%timeit splitter.split([text])[0]\n",
    "%timeit splitter.split([text] * 100)[0]\n",
    "%timeit splitter.split([text] * 1000)[0]\n",
    "%timeit splitter.split([text] * 10_000)[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "sTGnTBrqmAFh"
   },
   "source": [
    "And voilà! Splitting 10000 short texts in less than 400 milliseconds."
   ]
  }
 ],
 "metadata": {
  "accelerator": "GPU",
  "colab": {
   "collapsed_sections": [
    "tWGZJRWAl6w-"
   ],
   "provenance": [
    {
     "file_id": "https://github.com/bminixhofer/nnsplit/blob/master/train/train.ipynb",
     "timestamp": 1679216458728
    }
   ]
  },
  "gpuClass": "standard",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  },
  "widgets": {
   "application/vnd.jupyter.widget-state+json": {
    "0267c450243a42d08c5c0a6b6ebb3c43": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_348d359fec60427285df8e21a26c54a1",
      "placeholder": "​",
      "style": "IPY_MODEL_3e5ae13a018a471da030f3bfefa3337d",
      "value": " 0/196 [00:00&lt;?, ?it/s]"
     }
    },
    "0819be7a9cd8434484b6c7e62c33f6b5": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "0968a28f667d4410ab4504a8432af229": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ab43e58f2ae54f9d9ad243a9a08afd7b",
      "placeholder": "​",
      "style": "IPY_MODEL_a3c26d6b7d204e67b1f0cfb2291a0786",
      "value": " 9999999/? [48:22&lt;00:00, 3561.42it/s]"
     }
    },
    "103f112320b844498c1946cd1564bfee": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": "2",
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "1460af1ac54644edbaf21bcc5c197995": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_56c206cb7f77436684fcbfc3c7ff1abf",
      "placeholder": "​",
      "style": "IPY_MODEL_2deda6ac0ad243dbadc55c394b8f6baf",
      "value": ""
     }
    },
    "189110f7cf5d4595b5648c6bdcdec2ab": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "23374edc4a994c63a0abf9428dd41838": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "24661beeac614ce9873b3a8adbc8d428": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "2ca1fa79adeb47edbe57668b09c7bbed": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "danger",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_ea186d2a6dbe405d98f6b48769c671d4",
      "max": 1,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_189110f7cf5d4595b5648c6bdcdec2ab",
      "value": 1
     }
    },
    "2deda6ac0ad243dbadc55c394b8f6baf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "34667acf38a648a68f8af315b855ab18": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": "inline-flex",
      "flex": null,
      "flex_flow": "row wrap",
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "100%"
     }
    },
    "348d359fec60427285df8e21a26c54a1": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "397213b253854f2a9471401bcc51d4b4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": "2",
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "39ab6d5a11d4495e8cfeab49bf65fa9a": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "3e5ae13a018a471da030f3bfefa3337d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "3f7964262d2949f5ac0d45f640b1dfd1": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "5206d17ff7f14de6b8e313b59ee28f3b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_39ab6d5a11d4495e8cfeab49bf65fa9a",
      "placeholder": "​",
      "style": "IPY_MODEL_74a818e188604a9cbea85c43b0844d10",
      "value": "Validating:   0%"
     }
    },
    "52ea02224dee45c1b355be493c84b54f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_103f112320b844498c1946cd1564bfee",
      "max": 196,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_84642daaa6bd4f449ca05c0b51086ebf",
      "value": 0
     }
    },
    "56c206cb7f77436684fcbfc3c7ff1abf": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "60b5c2ee79c74150821be95c53fc43b2": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "60f1b29c27ff425c919327eb608a16c6": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_8f27cc99c5354e4f95330747277cc318",
       "IPY_MODEL_d15b9226cd384d8096aadccb8c948ae3",
       "IPY_MODEL_82044626fe8c4295b0e01a4145b8217b"
      ],
      "layout": "IPY_MODEL_34667acf38a648a68f8af315b855ab18"
     }
    },
    "74a818e188604a9cbea85c43b0844d10": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "82044626fe8c4295b0e01a4145b8217b": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_df17aed724bb4055968643aa9105aea9",
      "placeholder": "​",
      "style": "IPY_MODEL_ec63970793e945ecb1404930d6c6aaac",
      "value": " 0/2 [00:33&lt;?, ?it/s]"
     }
    },
    "83e0dbc5b6d44a729ac5d1180d037252": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_24661beeac614ce9873b3a8adbc8d428",
      "placeholder": "​",
      "style": "IPY_MODEL_87fb86503e1449dba9965267bf862425",
      "value": " 800/978 [16:16&lt;03:37,  1.22s/it, loss=0.0074, v_num=0]"
     }
    },
    "84642daaa6bd4f449ca05c0b51086ebf": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "87fb86503e1449dba9965267bf862425": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "8f27cc99c5354e4f95330747277cc318": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_23374edc4a994c63a0abf9428dd41838",
      "placeholder": "​",
      "style": "IPY_MODEL_3f7964262d2949f5ac0d45f640b1dfd1",
      "value": "Validation sanity check:   0%"
     }
    },
    "920f552635234c1f9f46c39f91c3ab2f": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_de35f8ce97194099a185915e3856dd1b",
      "max": 978,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_b6269e82c5a34514aeadc3f36ca7fd10",
      "value": 800
     }
    },
    "93dfab7bd9734a639f6f7622522b29fb": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "9dc970e0ffa840e5978c294f3e851d3d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_1460af1ac54644edbaf21bcc5c197995",
       "IPY_MODEL_2ca1fa79adeb47edbe57668b09c7bbed",
       "IPY_MODEL_0968a28f667d4410ab4504a8432af229"
      ],
      "layout": "IPY_MODEL_93dfab7bd9734a639f6f7622522b29fb"
     }
    },
    "9edde47996ef46c0b2739468d078746d": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "a3c26d6b7d204e67b1f0cfb2291a0786": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    },
    "ab43e58f2ae54f9d9ad243a9a08afd7b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ae03952b50a6425682d72f1b82669ebc": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_5206d17ff7f14de6b8e313b59ee28f3b",
       "IPY_MODEL_52ea02224dee45c1b355be493c84b54f",
       "IPY_MODEL_0267c450243a42d08c5c0a6b6ebb3c43"
      ],
      "layout": "IPY_MODEL_d1f339779e5943deaed85a3b9e6dd5cf"
     }
    },
    "b26c08b8114a4b8a96e981be7d6e46ca": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": "inline-flex",
      "flex": null,
      "flex_flow": "row wrap",
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "100%"
     }
    },
    "b6269e82c5a34514aeadc3f36ca7fd10": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "ProgressStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "ProgressStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "bar_color": null,
      "description_width": ""
     }
    },
    "d15b9226cd384d8096aadccb8c948ae3": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "FloatProgressModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "FloatProgressModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "ProgressView",
      "bar_style": "danger",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_397213b253854f2a9471401bcc51d4b4",
      "max": 2,
      "min": 0,
      "orientation": "horizontal",
      "style": "IPY_MODEL_9edde47996ef46c0b2739468d078746d",
      "value": 0
     }
    },
    "d1f339779e5943deaed85a3b9e6dd5cf": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": "inline-flex",
      "flex": null,
      "flex_flow": "row wrap",
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "100%"
     }
    },
    "db79ae604d284efb9984ec1004d1a460": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HTMLModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HTMLModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HTMLView",
      "description": "",
      "description_tooltip": null,
      "layout": "IPY_MODEL_60b5c2ee79c74150821be95c53fc43b2",
      "placeholder": "​",
      "style": "IPY_MODEL_0819be7a9cd8434484b6c7e62c33f6b5",
      "value": "Epoch 0:  82%"
     }
    },
    "dd749566875040c8890c6e270f8c3c34": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "HBoxModel",
     "state": {
      "_dom_classes": [],
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "HBoxModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/controls",
      "_view_module_version": "1.5.0",
      "_view_name": "HBoxView",
      "box_style": "",
      "children": [
       "IPY_MODEL_db79ae604d284efb9984ec1004d1a460",
       "IPY_MODEL_920f552635234c1f9f46c39f91c3ab2f",
       "IPY_MODEL_83e0dbc5b6d44a729ac5d1180d037252"
      ],
      "layout": "IPY_MODEL_b26c08b8114a4b8a96e981be7d6e46ca"
     }
    },
    "de35f8ce97194099a185915e3856dd1b": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": "2",
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "df17aed724bb4055968643aa9105aea9": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": null
     }
    },
    "ea186d2a6dbe405d98f6b48769c671d4": {
     "model_module": "@jupyter-widgets/base",
     "model_module_version": "1.2.0",
     "model_name": "LayoutModel",
     "state": {
      "_model_module": "@jupyter-widgets/base",
      "_model_module_version": "1.2.0",
      "_model_name": "LayoutModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "LayoutView",
      "align_content": null,
      "align_items": null,
      "align_self": null,
      "border": null,
      "bottom": null,
      "display": null,
      "flex": null,
      "flex_flow": null,
      "grid_area": null,
      "grid_auto_columns": null,
      "grid_auto_flow": null,
      "grid_auto_rows": null,
      "grid_column": null,
      "grid_gap": null,
      "grid_row": null,
      "grid_template_areas": null,
      "grid_template_columns": null,
      "grid_template_rows": null,
      "height": null,
      "justify_content": null,
      "justify_items": null,
      "left": null,
      "margin": null,
      "max_height": null,
      "max_width": null,
      "min_height": null,
      "min_width": null,
      "object_fit": null,
      "object_position": null,
      "order": null,
      "overflow": null,
      "overflow_x": null,
      "overflow_y": null,
      "padding": null,
      "right": null,
      "top": null,
      "visibility": null,
      "width": "20px"
     }
    },
    "ec63970793e945ecb1404930d6c6aaac": {
     "model_module": "@jupyter-widgets/controls",
     "model_module_version": "1.5.0",
     "model_name": "DescriptionStyleModel",
     "state": {
      "_model_module": "@jupyter-widgets/controls",
      "_model_module_version": "1.5.0",
      "_model_name": "DescriptionStyleModel",
      "_view_count": null,
      "_view_module": "@jupyter-widgets/base",
      "_view_module_version": "1.2.0",
      "_view_name": "StyleView",
      "description_width": ""
     }
    }
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}