Team_4_Assingment_2.ipynb

{
  "nbformat": 4,
  "nbformat_minor": 0,
  "metadata": {
    "colab": {
      "name": "Team_4_Assingment_2.ipynb",
      "provenance": [],
      "collapsed_sections": [],
      "toc_visible": true
    },
    "kernelspec": {
      "name": "python3",
      "display_name": "Python 3"
    },
    "language_info": {
      "name": "python"
    }
  },
  "cells": [
    {
      "cell_type": "markdown",
      "source": [
        "#Team Information & Contents"
      ],
      "metadata": {
        "id": "44Y7JUerJh7j"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "\n",
        "|Team-number :| 4|\n",
        "|:----:|:----:|\n",
        "\n",
        "\n",
        "|Name|    E-Mail        |matriculation-nr.|\n",
        "|:----:|:----:|:----:|\n",
        "|Altuğ Demirşah|demira02@ads.uni-passau.de|109776|\n",
        "|Elif Günay| \tguenay01@ads.uni-passau.de|109806|\n",
        "|Evren Can| can05@ads.uni-passau.de|109484|\n",
        "\n",
        "\n",
        "**\n",
        "\n",
        "--------------------------------------------------------------------------------\n",
        "\n",
        "**\n",
        "\n",
        "This report is provided for the second-assignment of the Deep Learning and Text Analysis in Finance course at the University of Passau on WiSe 21/22.\n",
        "\n",
        "**\n",
        "\n",
        "**Aim**: This report contains our own preprocessing and modelling approaches for a text input that labeled by us on label studio in order to predict its labels.\n",
        "\n",
        "**\n",
        "\n",
        "**Contents**:\n",
        "\n",
        "1. **Import Needed Libraries**\n",
        "\n",
        "2. **Data Preprocessing**\n",
        "\n",
        "    2.1. Generate a Dataframe\n",
        "\n",
        "    2.2. Generate Label\n",
        "\n",
        "    2.3. Generate Corpus\n",
        "\n",
        "3. **Topic Modeling**\n",
        "\n",
        "    3.1. Convert to Document-Term Matrix\n",
        "\n",
        "    3.2. Convert to Frequency-Inverse Document Frequency Matrix\n",
        "\n",
        "    3.3. Fit Latent Dirichet Allocation Models\n",
        "\n",
        "\n",
        "4. **Doc2Vec & Embedding**\n",
        "\n",
        "    4.1. Doc2Vec\n",
        "\n",
        "      4.1.2 Why we use Doc2Vec instead of Word2Vec?\n",
        "\n",
        "    4.2. Embedding\n",
        "\n",
        "5. **Modeling**\n",
        "\n",
        "    5.1. Generate Evaluation Metrics and Model Checkpoint\n",
        "\n",
        "    5.2. Train-Test Split\n",
        "\n",
        "    5.3. Architecture\n",
        "    \n",
        "      5.3.1. Why did we use RNN?\n",
        "\n",
        "    5.4. Prediction\n",
        "\n",
        "\n",
        "6. **Comments**\n",
        "\n",
        "    6.1. Reparding to data preprocessing\n",
        "\n",
        "    6.2. Regarding to topic modeling\n",
        "\n",
        "    6.3. Reparding to doc2vec & embedding \n",
        "\n",
        "    6.4. Final comments\n",
        "\n",
        "\n",
        "In order to run this code file, these should be considered:\n",
        "\n",
        "- This code file was created on Google Colab and data was uploaded from Google Drive.\n",
        "- Since we run this code file on the basic Colab version, limited memory and computational sources are provided by the platform. Running this code file on a different device might require/take a long time depending on the sources that the device has."
      ],
      "metadata": {
        "id": "WjBieQSYJm-D"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "#Import needed libraries"
      ],
      "metadata": {
        "id": "U8hrNdjaLBm8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "Needed libraries:\n",
        "\n",
        "  For data proprocessing:\n",
        "\n",
        "    pandas: we use pandas library to manipulate data and analyse inside of it. In our case, manipulating numerical tables and time series.\n",
        "    numpy:  we use numpy since it offers a large collection of high-level mathematical functions to operate on the arrays.\n",
        "    matplotlib: matplotlib is a plotting library\n",
        "    sklearn:  to change raw feature vectors into a representation that is more suitable for the downstream estimators\n",
        "\n",
        "    re -> Regular expression operations\n",
        "    nltk -> is a suite of libraries and programs for symbolic and statistical natural language processing for English written in the Python programming language.\n",
        "    gensim -> open-source library for unsupervised topic modeling and natural language processing\n",
        "\n",
        "    pyLDAvis -> designed to help users interpret the topics in a topic model that has been fit to a corpus of text data\n",
        "\n",
        "  For modelling:\n",
        "  \n",
        "    tensorflow:  used to design, build, and train deep learning models. \n",
        "      ->  layers: this is the class from which all layers inherit such as LSTM, GRU, Dense and so on\n",
        "   \n",
        "\"\"\"\n",
        "\n",
        "# data preprocessing\n",
        "import numpy as np\n",
        "import pandas as pd \n",
        "import pickle\n",
        "import matplotlib.pylab as plt\n",
        "\n",
        "import re\n",
        "import string\n",
        "import nltk\n",
        "nltk.download('punkt')\n",
        "from nltk import word_tokenize\n",
        "import gensim\n",
        "from gensim.models.doc2vec import Doc2Vec\n",
        "from gensim.parsing.preprocessing import STOPWORDS\n",
        "\n",
        "from sklearn.model_selection import train_test_split\n",
        "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n",
        "from sklearn.decomposition import LatentDirichletAllocation as LDA_model\n",
        "\n",
        "!pip install pyLDAvis\n",
        "import pyLDAvis        \n",
        "import pyLDAvis.sklearn\n",
        "pyLDAvis.enable_notebook()\n",
        "\n",
        "\n",
        "# modelling\n",
        "import tensorflow as tf\n",
        "from keras.preprocessing.text import one_hot\n",
        "from keras.preprocessing.sequence import pad_sequences\n",
        "from keras.models import Sequential\n",
        "from keras.layers import Dense, Flatten, GRU, LSTM, Embedding\n",
        "from keras import backend as K\n",
        "from keras.callbacks import ModelCheckpoint"
      ],
      "metadata": {
        "id": "au3NH-wOLAyo",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "55cabd71-60c8-461e-ac8d-70cf2f3f0466"
      },
      "execution_count": 1,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "[nltk_data] Downloading package punkt to /root/nltk_data...\n",
            "[nltk_data]   Package punkt is already up-to-date!\n",
            "Requirement already satisfied: pyLDAvis in /usr/local/lib/python3.7/dist-packages (3.3.1)\n",
            "Requirement already satisfied: gensim in /usr/local/lib/python3.7/dist-packages (from pyLDAvis) (3.6.0)\n",
            "Requirement already satisfied: setuptools in /usr/local/lib/python3.7/dist-packages (from pyLDAvis) (57.4.0)\n",
            "Requirement already satisfied: sklearn in /usr/local/lib/python3.7/dist-packages (from pyLDAvis) (0.0)\n",
            "Requirement already satisfied: future in /usr/local/lib/python3.7/dist-packages (from pyLDAvis) (0.16.0)\n",
            "Requirement already satisfied: joblib in /usr/local/lib/python3.7/dist-packages (from pyLDAvis) (1.1.0)\n",
            "Requirement already satisfied: scipy in /usr/local/lib/python3.7/dist-packages (from pyLDAvis) (1.4.1)\n",
            "Requirement already satisfied: numpy>=1.20.0 in /usr/local/lib/python3.7/dist-packages (from pyLDAvis) (1.21.5)\n",
            "Requirement already satisfied: scikit-learn in /usr/local/lib/python3.7/dist-packages (from pyLDAvis) (1.0.2)\n",
            "Requirement already satisfied: numexpr in /usr/local/lib/python3.7/dist-packages (from pyLDAvis) (2.8.1)\n",
            "Requirement already satisfied: jinja2 in /usr/local/lib/python3.7/dist-packages (from pyLDAvis) (2.11.3)\n",
            "Requirement already satisfied: pandas>=1.2.0 in /usr/local/lib/python3.7/dist-packages (from pyLDAvis) (1.3.5)\n",
            "Requirement already satisfied: funcy in /usr/local/lib/python3.7/dist-packages (from pyLDAvis) (1.17)\n",
            "Requirement already satisfied: pytz>=2017.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.2.0->pyLDAvis) (2018.9)\n",
            "Requirement already satisfied: python-dateutil>=2.7.3 in /usr/local/lib/python3.7/dist-packages (from pandas>=1.2.0->pyLDAvis) (2.8.2)\n",
            "Requirement already satisfied: six>=1.5 in /usr/local/lib/python3.7/dist-packages (from python-dateutil>=2.7.3->pandas>=1.2.0->pyLDAvis) (1.15.0)\n",
            "Requirement already satisfied: smart-open>=1.2.1 in /usr/local/lib/python3.7/dist-packages (from gensim->pyLDAvis) (5.2.1)\n",
            "Requirement already satisfied: MarkupSafe>=0.23 in /usr/local/lib/python3.7/dist-packages (from jinja2->pyLDAvis) (2.0.1)\n",
            "Requirement already satisfied: packaging in /usr/local/lib/python3.7/dist-packages (from numexpr->pyLDAvis) (21.3)\n",
            "Requirement already satisfied: pyparsing!=3.0.5,>=2.0.2 in /usr/local/lib/python3.7/dist-packages (from packaging->numexpr->pyLDAvis) (3.0.7)\n",
            "Requirement already satisfied: threadpoolctl>=2.0.0 in /usr/local/lib/python3.7/dist-packages (from scikit-learn->pyLDAvis) (3.1.0)\n"
          ]
        },
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/past/types/oldstr.py:5: DeprecationWarning: Using or importing the ABCs from 'collections' instead of from 'collections.abc' is deprecated since Python 3.3,and in 3.9 it will stop working\n",
            "  from collections import Iterable\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#Connect google drive & load data"
      ],
      "metadata": {
        "id": "JahcJpI0LYFs"
      }
    },
    {
      "cell_type": "code",
      "execution_count": 2,
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "tsCn5DkXWVvm",
        "outputId": "f7d761a5-01ab-4b21-9c12-b8b32dcb8728"
      },
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Drive already mounted at /gdrive; to attempt to forcibly remount, call drive.mount(\"/gdrive\", force_remount=True).\n"
          ]
        }
      ],
      "source": [
        "# to download data, first we need to connect to google drive\n",
        "from google.colab import drive\n",
        "drive.mount('/gdrive')\n",
        "\n",
        "\n",
        "# load data\n",
        "path = '/gdrive/MyDrive/DLTA/Data/tasks3'\n",
        "with open(path, 'rb') as f:\n",
        "    tasks = pickle.load(f)"
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#Data preprocessing\n",
        "\n",
        "* Before we do anything, text must be preprocessed. This can be tricky for certain documents, but,  for the moment we will use an easy self-made implementation of some existing functions which will do an effecient progress for the text preprocessing.\n",
        "\n",
        "* Typical steps of preprocessing include exclusion of punctuation, the transformation to lower case letters, the removal of numbers and more steps such as the removal of stopwords and stemming. For instance;\n",
        "\n",
        "* Stopwords: A stop word is a commonly used word (such as “the”, “a”, “an”, “in”). We would not want these words to take up space in our dataset, or taking up valuable processing time. Also, when looking for common words in our dataset, finding out that \"a\" is the most common, is not particularly interesting\n",
        "\n",
        "* Lemmatising: Lemmatisation (or lemmatization) in linguistics is the process of grouping together the inflected forms of a word so they can be analysed as a single item, identified by the word's lemma, or dictionary form.\n",
        "\n",
        "    Loved -> love\n",
        "\n",
        "* Stemming: In linguistic morphology and information retrieval, stemming is the process of reducing inflected (or sometimes derived) words to their word stem, base or root form—generally a written word form\n",
        "\n",
        "    Loved -> lov"
      ],
      "metadata": {
        "id": "J4wFzqdQLjKM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "##Generate a dataframe"
      ],
      "metadata": {
        "id": "9RdlfsNeMNF8"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def generate_df(tasks : list, index : int) -> pd.DataFrame:\n",
        "  \"\"\"\"\n",
        "  generate a dataframe in order to keep data clean and work with\n",
        "\n",
        "  Arg:\n",
        "    tasks: the dataset\n",
        "    index: index of annotation between 62, 63, 64\n",
        "  \n",
        "  Return:\n",
        "    pd.DataFrame\n",
        "  \n",
        "  \"\"\"\n",
        "  \n",
        "  #############################################################################################################################################\n",
        "  # code below is taken from Mohammed's post on stud ip\n",
        "\n",
        "  df = pd.DataFrame(columns=[\"text\", \"category\", \"stage\", \"level\"])\n",
        "\n",
        "  question_tags = [\n",
        "        \"Question_1_Company_specific\", \"Question_1_Market_related\",\n",
        "        \"Question_2_specific\", \"Question_2_open\", \n",
        "        \"Question_3_attack\", \"Question_3_support\", \"Question_3_neutral\"\n",
        "  ]\n",
        "\n",
        "  answer_tags = [\n",
        "        \"Answer_1_specific\", \"Answer_1_avoid_excuse\", \n",
        "        \"Answer_2_negative\", \"Answer_2_positive\", \n",
        "        \"Answer_3_blame\", \"Answer_3_no_blame\"\n",
        "  ]\n",
        "\n",
        "  tag2val = {\n",
        "        \"Question_1_Company_specific\": 0,\n",
        "        \"Question_1_Market_related\": 1,\n",
        "        \"Question_2_specific\": 0,\n",
        "        \"Question_2_open\": 1,\n",
        "        \"Question_3_attack\": 0,\n",
        "        \"Question_3_support\": 1,\n",
        "        \"Question_3_neutral\": 2,\n",
        "        \"Answer_1_specific\": 0,\n",
        "        \"Answer_1_avoid_excuse\": 1,\n",
        "        \"Answer_2_negative\": 0,\n",
        "        \"Answer_2_positive\": 1,\n",
        "        \"Answer_3_blame\": 0,\n",
        "        \"Answer_3_no_blame\": 1\n",
        "  }\n",
        "\n",
        "  raw_data = tasks[index]\n",
        "\n",
        "  annotations = raw_data[\"annotations\"][0][\"result\"]\n",
        "\n",
        "  for ann in annotations:\n",
        "            text = ann[\"value\"][\"text\"]\n",
        "            label = ann[\"value\"][\"labels\"][0]\n",
        "            \n",
        "\n",
        "            # Ignore any odservation that does not have any of the question and answer stages\n",
        "            # (ex: an observation that only labels the question QID_13)\n",
        "            if len(label) < 7:\n",
        "                continue\n",
        "\n",
        "            # Get the category \"Answer\" or \"Question\" from the label\n",
        "            category = \"Answer\" if \"Answer\" in label else \"Question\"\n",
        "            # Get the stage number\n",
        "            stage = int(label.split(\"_\")[1])\n",
        "            # Get the level\n",
        "            level = tag2val[label]\n",
        "            # Add the observation to the dataframe\n",
        "            df = df.append(\n",
        "                {\n",
        "                    \"text\": text,\n",
        "                    \"category\": category,\n",
        "                    \"stage\": stage,\n",
        "                    \"level\": level\n",
        "                },\n",
        "                ignore_index=True)    \n",
        "\n",
        "  return df\n",
        "\n",
        "#############################################################################################################################################\n",
        "\n",
        "# our earning call ids: 62, 63 and 64 so any of them can be chosen and work with\n",
        "# since the earling call which is represented with the id of 62 contains more data than the others,\n",
        "# so we prefered working only with this earning call to focus on data preprocessing and modelling part of NLP, not repating same steps on each earning calls\n",
        "df = generate_df(tasks, 62)\n",
        "\n",
        "# check all rows and colomns in df in order to avoid data leakage\n",
        "pd.set_option('display.max_rows', df.shape[0]+1)\n",
        "print(df)"
      ],
      "metadata": {
        "id": "kvzOr9AkYT4e",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "390bd2a2-e42d-4885-f432-67993d01ca93"
      },
      "execution_count": 3,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "                                                  text  category stage level\n",
            "0     I know you don't want to be too specific arou...  Question     1     1\n",
            "1     I know you don't want to be too specific arou...  Question     2     0\n",
            "2     I know you don't want to be too specific arou...  Question     3     2\n",
            "3     what we're seeing in memory is customers cont...    Answer     1     0\n",
            "4    what we're seeing in memory is customers conti...    Answer     2     1\n",
            "5    what we're seeing in memory is customers conti...    Answer     3     1\n",
            "6    If I could sneak in a quick follow-up, like yo...  Question     1     0\n",
            "7    If I could sneak in a quick follow-up, like yo...  Question     2     0\n",
            "8    If I could sneak in a quick follow-up, like yo...  Question     3     1\n",
            "9    the services business is a great growth driver...    Answer     1     0\n",
            "10   the services business is a great growth driver...    Answer     2     1\n",
            "11   the services business is a great growth driver...    Answer     3     1\n",
            "12   can you talk about the puts and takes between ...  Question     1     0\n",
            "13   can you talk about the puts and takes between ...  Question     2     0\n",
            "14    can you talk about the puts and takes between...  Question     3     2\n",
            "15   I'll actually take both parts of your question...    Answer     1     0\n",
            "16    I'll actually take both parts of your questio...    Answer     2     1\n",
            "17   I'll actually take both parts of your question...    Answer     3     1\n",
            "18   Gary, you talked quite a bit about your expect...  Question     1     1\n",
            "19   Gary, you talked quite a bit about your expect...  Question     2     1\n",
            "20   Gary, you talked quite a bit about your expect...  Question     3     2\n",
            "21   the percentage of the addressable market that ...    Answer     1     1\n",
            "22   the percentage of the addressable market that ...    Answer     2     1\n",
            "23   the percentage of the addressable market that ...    Answer     3     1\n",
            "24    I think relative to the different areas where...    Answer     1     0\n",
            "25   I think relative to the different areas where ...    Answer     2     1\n",
            "26   I think relative to the different areas where ...    Answer     3     1\n",
            "27   can you give us an update on the Kokusai acqui...  Question     2     1\n",
            "28   can you give us an update on the Kokusai acqui...  Question     3     0\n",
            "29    transition to Gary on the second part. For Ko...    Answer     1     1\n",
            "30   Toshiya relative to Kokusai, we still see it a...    Answer     1     1\n",
            "31   transition to Gary on the second part. For Kok...    Answer     2     1\n",
            "32   Toshiya relative to Kokusai, we still see it a...    Answer     2     1\n",
            "33   transition to Gary on the second part. For Kok...    Answer     3     1\n",
            "34   Toshiya relative to Kokusai, we still see it a...    Answer     3     1\n",
            "35   I think in the past you guys have spoken about...  Question     1     0\n",
            "36   I think in the past you guys have spoken about...  Question     2     0\n",
            "37   I think in the past you guys have spoken about...  Question     3     1\n",
            "38   If in 2020, if everything else being equal, if...  Question     1     0\n",
            "39   If in 2020, if everything else being equal, if...  Question     2     0\n",
            "40   If in 2020, if everything else being equal, if...  Question     3     1\n",
            "41   From a WFE standpoint, we don't want to get in...    Answer     1     1\n",
            "42   From a WFE standpoint, we don't want to get in...    Answer     2     0\n",
            "43   From a WFE standpoint, we don't want to get in...    Answer     3     1\n",
            "44   you guys have done a great job to kind of talk...  Question     1     0\n",
            "45   you guys have done a great job to kind of talk...  Question     2     1\n",
            "46   you guys have done a great job to kind of talk...  Question     3     1\n",
            "47   Our view on China is pretty similar to what we...    Answer     1     1\n",
            "48   As you think about our planning, we always hav...    Answer     1     1\n",
            "49   Our view on China is pretty similar to what we...    Answer     2     1\n",
            "50   As you think about our planning, we always hav...    Answer     2     1\n",
            "51   Our view on China is pretty similar to what we...    Answer     3     1\n",
            "52   As you think about our planning, we always hav...    Answer     3     1\n",
            "53   Then as my follow up Gary, just going back to ...  Question     1     1\n",
            "54   Then as my follow up Gary, just going back to ...  Question     2     1\n",
            "55   Then as my follow up Gary, just going back to ...  Question     3     0\n",
            "56   Let me jump in on that and give you my perspec...    Answer     1     1\n",
            "57   Let me jump in on that and give you my perspec...    Answer     2     1\n",
            "58   Let me jump in on that and give you my perspec...    Answer     3     1\n",
            "59   On the better performance in AGS in the Octobe...  Question     1     1\n",
            "60   On the better performance in AGS in the Octobe...  Question     2     0\n",
            "61   On the better performance in AGS in the Octobe...  Question     3     2\n",
            "62    You're right. The long-term service agreement...    Answer     1     1\n",
            "63   You're right. The long-term service agreement ...    Answer     2     1\n",
            "64   You're right. The long-term service agreement ...    Answer     3     1\n",
            "65   Yes, and I think the big needle mover is reall...    Answer     1     1\n",
            "66   Yes, and I think the big needle mover is reall...    Answer     2     1\n",
            "67   Yes, and I think the big needle mover is reall...    Answer     3     1\n",
            "68   And when we think between cost per bit decline...  Question     1     0\n",
            "69   And when we think between cost per bit decline...  Question     2     0\n",
            "70   And when we think between cost per bit decline...  Question     3     2\n",
            "71   I talked earlier about co-optimizing hard mask...    Answer     1     0\n",
            "72   I talked earlier about co-optimizing hard mask...    Answer     2     1\n",
            "73   I talked earlier about co-optimizing hard mask...    Answer     3     1\n",
            "74   Could you give us -- I have heard with a lot o...  Question     1     0\n",
            "75   Could you give us -- I have heard with a lot o...  Question     3     1\n",
            "76   Yes. Thanks, Pierre. So, it’s important to rem...    Answer     1     0\n",
            "77   Yes. Thanks, Pierre. So, it’s important to rem...    Answer     2     1\n",
            "78   Yes. Thanks, Pierre. So, it’s important to rem...    Answer     3     1\n",
            "79   Gary, if think you just provided a little bit ...  Question     1     1\n",
            "80   Gary, if think you just provided a little bit ...  Question     2     1\n",
            "81   Gary, if think you just provided a little bit ...  Question     3     2\n",
            "82   if we look at 2019, we talked about a record f...    Answer     1     1\n",
            "83   if we look at 2019, we talked about a record f...    Answer     2     1\n",
            "84   if we look at 2019, we talked about a record f...    Answer     3     1\n",
            "85   I just had a question on gross margin guidance...  Question     1     0\n",
            "86   I just had a question on gross margin guidance...  Question     2     0\n",
            "87   I just had a question on gross margin guidance...  Question     3     0\n",
            "88   We won't guide gross margins, but let me -- by...    Answer     1     0\n",
            "89   We won't guide gross margins, but let me -- by...    Answer     2     0\n",
            "90    We won't guide gross margins, but let me -- b...    Answer     3     0\n",
            "91   Just thinking conceptually about next year, if...  Question     1     1\n",
            "92    Just thinking conceptually about next year, i...  Question     2     1\n",
            "93   Just thinking conceptually about next year, if...  Question     3     1\n",
            "94    if memory spending is incremental to the mode...  Question     1     1\n",
            "95    if memory spending is incremental to the mode...  Question     2     1\n",
            "96    if memory spending is incremental to the mode...  Question     3     1\n",
            "97   I guess, the best way for me to describe it, b...    Answer     1     1\n",
            "98   I guess, the best way for me to describe it, b...    Answer     2     0\n",
            "99   I guess, the best way for me to describe it, b...    Answer     3     0\n",
            "100  And I guess I just want to follow-up on John, ...  Question     1     1\n",
            "101  And I guess I just want to follow-up on John, ...  Question     2     1\n",
            "102  And I guess I just want to follow-up on John, ...  Question     3     0\n",
            "103  We've got a nice backlog, Q4 was a really nice...    Answer     1     1\n",
            "104  We've got a nice backlog, Q4 was a really nice...    Answer     2     0\n",
            "105  We've got a nice backlog, Q4 was a really nice...    Answer     3     1\n",
            "106  can you give us an update on the Kokusai acqui...  Question     1     0\n",
            "107  Could you give us -- I have heard with a lot o...  Question     2     0\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "##Generate label"
      ],
      "metadata": {
        "id": "yEQHJ2grT01n"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def choose_label(df : pd.DataFrame, category : str, stage : str) -> pd.DataFrame:\n",
        "  \"\"\"\n",
        "  This function allows us to create a new label dataframe in order to work with chosen category, stage and to avoid text repeating\n",
        "\n",
        "  Arg:\n",
        "    df (pd.DataFrame): a dataframe that will be worked on\n",
        "    category: category that will be picked beetween 'Question' and 'Answer'\n",
        "    stage: stage that will be picked between 1, 2, 3(only for Question_3)\n",
        "\n",
        "  Returns:\n",
        "    pd.DataFrame\n",
        "  \n",
        "  \"\"\"\n",
        "  \n",
        "\n",
        "  label_df = df.loc[(df['category'] == category) & (df['stage'] == stage)].reset_index(drop=True)\n",
        "\n",
        "  return label_df\n",
        "\n",
        "\n",
        "# any category('Question' or 'Answer') and stage (1,2 or 3(can be chosen only for Question_3)) can be chosen\n",
        "# since Question_1's distribution is balanced, the same amount of positive and negative samples, we work with that category to maximize the performance\n",
        "label_df = choose_label(df, 'Question', 1)\n",
        "label_df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 582
        },
        "id": "LGLBiZCpORGh",
        "outputId": "3e791e80-4261-4abb-8941-c1ec5776bcc3"
      },
      "execution_count": 4,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                                 text  category stage level\n",
              "0    I know you don't want to be too specific arou...  Question     1     1\n",
              "1   If I could sneak in a quick follow-up, like yo...  Question     1     0\n",
              "2   can you talk about the puts and takes between ...  Question     1     0\n",
              "3   Gary, you talked quite a bit about your expect...  Question     1     1\n",
              "4   I think in the past you guys have spoken about...  Question     1     0\n",
              "5   If in 2020, if everything else being equal, if...  Question     1     0\n",
              "6   you guys have done a great job to kind of talk...  Question     1     0\n",
              "7   Then as my follow up Gary, just going back to ...  Question     1     1\n",
              "8   On the better performance in AGS in the Octobe...  Question     1     1\n",
              "9   And when we think between cost per bit decline...  Question     1     0\n",
              "10  Could you give us -- I have heard with a lot o...  Question     1     0\n",
              "11  Gary, if think you just provided a little bit ...  Question     1     1\n",
              "12  I just had a question on gross margin guidance...  Question     1     0\n",
              "13  Just thinking conceptually about next year, if...  Question     1     1\n",
              "14   if memory spending is incremental to the mode...  Question     1     1\n",
              "15  And I guess I just want to follow-up on John, ...  Question     1     1\n",
              "16  can you give us an update on the Kokusai acqui...  Question     1     0"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-558876d6-7f31-4f5a-b5bc-c5767bf398dc\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>text</th>\n",
              "      <th>category</th>\n",
              "      <th>stage</th>\n",
              "      <th>level</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>I know you don't want to be too specific arou...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>If I could sneak in a quick follow-up, like yo...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>can you talk about the puts and takes between ...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Gary, you talked quite a bit about your expect...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>I think in the past you guys have spoken about...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>If in 2020, if everything else being equal, if...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>you guys have done a great job to kind of talk...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>Then as my follow up Gary, just going back to ...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>On the better performance in AGS in the Octobe...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>And when we think between cost per bit decline...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>Could you give us -- I have heard with a lot o...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>Gary, if think you just provided a little bit ...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>I just had a question on gross margin guidance...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>Just thinking conceptually about next year, if...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>if memory spending is incremental to the mode...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>And I guess I just want to follow-up on John, ...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>can you give us an update on the Kokusai acqui...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-558876d6-7f31-4f5a-b5bc-c5767bf398dc')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-558876d6-7f31-4f5a-b5bc-c5767bf398dc button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-558876d6-7f31-4f5a-b5bc-c5767bf398dc');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 4
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Generate corpus"
      ],
      "metadata": {
        "id": "i-IVQS1QXrRB"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def text_preprocessor(text_to_clean: list, remove_stopwords = False, stemming = False) -> list:\n",
        "    \"\"\"\n",
        "    preprocessing a text with applying lower case, removing numbers, puntuations, white spaces and stopwords in order to strength text input\n",
        "\n",
        "    Arg:\n",
        "      text_to_clean(list): text inputs that will be preprocessed\n",
        "      remove_stopwords(bool): True or False, default False\n",
        "      stemming(bool): process of reducing inflected words to their word stem, base or root form, True or False, default False\n",
        "\n",
        "    Return:\n",
        "      preprocessed text\n",
        "    \n",
        "    \"\"\"\n",
        "\n",
        "    # import a stemmer for english words\n",
        "    snowStem = nltk.stem.SnowballStemmer('english')\n",
        "    # lower case\n",
        "    text_to_clean = text_to_clean.lower()\n",
        "    # remove numbers \n",
        "    text_to_clean = re.sub(r'\\d+', '', text_to_clean)\n",
        "    # remove punctuation\n",
        "    text_to_clean = text_to_clean.translate(str.maketrans('','', string.punctuation))\n",
        "    # remove leading and ending white spaces\n",
        "    text_to_clean = text_to_clean.strip()\n",
        "    \n",
        "    if stemming:\n",
        "      # stemming\n",
        "      text_to_clean = snowStem.stem(text_to_clean)\n",
        "    \n",
        "    if remove_stopwords:\n",
        "        # remove stop words\n",
        "        text_to_clean = ' '.join([w for w in text_to_clean.split() if not(w in STOPWORDS)])\n",
        "    \n",
        "    return text_to_clean\n",
        "\n",
        "preprocessed_data = []\n",
        "\n",
        "for text in label_df['text']:\n",
        "  preprocessed_data.append(text_preprocessor(text, remove_stopwords=True, stemming=False))\n",
        "\n",
        "label_df['clean_text'] = preprocessed_data\n",
        "label_df"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 582
        },
        "id": "dEahCgzoXHSC",
        "outputId": "f4ae31c7-38dd-4799-c17a-a0e7557a6cf6"
      },
      "execution_count": 5,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "                                                 text  category stage level  \\\n",
              "0    I know you don't want to be too specific arou...  Question     1     1   \n",
              "1   If I could sneak in a quick follow-up, like yo...  Question     1     0   \n",
              "2   can you talk about the puts and takes between ...  Question     1     0   \n",
              "3   Gary, you talked quite a bit about your expect...  Question     1     1   \n",
              "4   I think in the past you guys have spoken about...  Question     1     0   \n",
              "5   If in 2020, if everything else being equal, if...  Question     1     0   \n",
              "6   you guys have done a great job to kind of talk...  Question     1     0   \n",
              "7   Then as my follow up Gary, just going back to ...  Question     1     1   \n",
              "8   On the better performance in AGS in the Octobe...  Question     1     1   \n",
              "9   And when we think between cost per bit decline...  Question     1     0   \n",
              "10  Could you give us -- I have heard with a lot o...  Question     1     0   \n",
              "11  Gary, if think you just provided a little bit ...  Question     1     1   \n",
              "12  I just had a question on gross margin guidance...  Question     1     0   \n",
              "13  Just thinking conceptually about next year, if...  Question     1     1   \n",
              "14   if memory spending is incremental to the mode...  Question     1     1   \n",
              "15  And I guess I just want to follow-up on John, ...  Question     1     1   \n",
              "16  can you give us an update on the Kokusai acqui...  Question     1     0   \n",
              "\n",
              "                                           clean_text  \n",
              "0   know dont want specific timing magnitude recov...  \n",
              "1   sneak quick followup like said quick second se...  \n",
              "2   talk puts takes oled lcd kind flattish display...  \n",
              "3   gary talked bit expectations new products year...  \n",
              "4   think past guys spoken data analytics talking ...  \n",
              "5   equal nand wfe putting numbers expect amat sem...  \n",
              "6   guys great job kind talking trends relative de...  \n",
              "7   follow gary going comments foundry logic looki...  \n",
              "8   better performance ags october quarter strong ...  \n",
              "9   think cost bit declines memory dram nand rate ...  \n",
              "10  heard lot answers you’ve given outlook logic t...  \n",
              "11  gary think provided little bit color question ...  \n",
              "12  question gross margin guidance theres hardly i...  \n",
              "13  thinking conceptually year foundry logic susta...  \n",
              "14  memory spending incremental model said foundry...  \n",
              "15  guess want followup john questions illogical a...  \n",
              "16  update kokusai acquisition regulatory hurdles ...  "
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-516f830a-0b8f-48b9-972a-902acfe7e62a\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>text</th>\n",
              "      <th>category</th>\n",
              "      <th>stage</th>\n",
              "      <th>level</th>\n",
              "      <th>clean_text</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>I know you don't want to be too specific arou...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>know dont want specific timing magnitude recov...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>If I could sneak in a quick follow-up, like yo...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>sneak quick followup like said quick second se...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>can you talk about the puts and takes between ...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>talk puts takes oled lcd kind flattish display...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>Gary, you talked quite a bit about your expect...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>gary talked bit expectations new products year...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>I think in the past you guys have spoken about...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>think past guys spoken data analytics talking ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>If in 2020, if everything else being equal, if...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>equal nand wfe putting numbers expect amat sem...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>you guys have done a great job to kind of talk...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>guys great job kind talking trends relative de...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>Then as my follow up Gary, just going back to ...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>follow gary going comments foundry logic looki...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>On the better performance in AGS in the Octobe...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>better performance ags october quarter strong ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>And when we think between cost per bit decline...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>think cost bit declines memory dram nand rate ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>Could you give us -- I have heard with a lot o...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>heard lot answers you’ve given outlook logic t...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>Gary, if think you just provided a little bit ...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>gary think provided little bit color question ...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>I just had a question on gross margin guidance...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>question gross margin guidance theres hardly i...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>Just thinking conceptually about next year, if...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>thinking conceptually year foundry logic susta...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>if memory spending is incremental to the mode...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>memory spending incremental model said foundry...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>And I guess I just want to follow-up on John, ...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>guess want followup john questions illogical a...</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>can you give us an update on the Kokusai acqui...</td>\n",
              "      <td>Question</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>update kokusai acquisition regulatory hurdles ...</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-516f830a-0b8f-48b9-972a-902acfe7e62a')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-516f830a-0b8f-48b9-972a-902acfe7e62a button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-516f830a-0b8f-48b9-972a-902acfe7e62a');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 5
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Topic Modelling\n",
        "\n",
        "is an unsupervised machine learning technique that can be applied to process a large number of unlabeled text documents to discover hidden topics.\n",
        "\n",
        "Common topic modelling algorithms:\n",
        "\n",
        " - *Latent Semantic Analysis*(LSA): Matrix decomposition\n",
        "\n",
        " - *Latent Dirichlet Allocation*(LDA): probabilistic inference\n",
        "\n",
        "LSA vs LDA: same input, similar output, different math\n",
        "\n",
        " - first step is to convert text into numerical representation such as document term frequency matrix that contains the word count for each word in each document and these words are called term and this matrix is also known as corpus\n",
        " \n",
        " - with **LSA**, after you apply singular value decomposition(svm) and you obtain two output matrices\n",
        " \n",
        "  - document spesific topic allocation matrix\n",
        "  - topic spesific word allocation matrix\n",
        " \n",
        " - with **LDA**, which is a particularly popular method for fitting a topic model, you apply statistical inference to obtain similar output but in the form of probability disturabion \n",
        "  \n",
        "  - given a document what is the probability distibution of each topic with in the document\n",
        "  \n",
        "  - given a specific topic what is your word distibution from your vocabulary \n",
        "\n",
        " - ones you have these outputs, important point is such as document similarity and word similarity can be calculated; for example based on the dot product of corresponding rows"
      ],
      "metadata": {
        "id": "sOOIeNt8QKgB"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Convert to document-term matrix"
      ],
      "metadata": {
        "id": "rfdj6DOYfVJ1"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# Convert a collection of text documents to a matrix of token counts\n",
        "\n",
        "# term frequency vector\n",
        "tf_vectorizer = CountVectorizer(strip_accents = 'unicode',            # remove accents and perform other character normalization during the preprocessing step\n",
        "                                stop_words = 'english',               \n",
        "                                lowercase = True,                     # convert all characters to lowercase before tokenizing\n",
        "                                token_pattern = r'\\b[a-zA-Z]{3,}\\b'   # choose the words contain letters, not numbers, chosen words need to have at least three letters\n",
        "                              )\n",
        "\n",
        "# document term frequency matrix\n",
        "dtm_tf = tf_vectorizer.fit_transform(label_df['text'])       # label_df['text'] performs slightly better than label_df['clean_text']\n",
        "pd.DataFrame(data = dtm_tf.toarray(), columns = tf_vectorizer.get_feature_names())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 667
        },
        "id": "qDa92l6-ShYJ",
        "outputId": "09d03702-96ae-4dbc-f86a-b402b3aad28e"
      },
      "execution_count": 6,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function get_feature_names is deprecated; get_feature_names is deprecated in 1.0 and will be removed in 1.2. Please use get_feature_names_out instead.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "    acquisition  actually  additional  ags  amat  analytics  answers  applied  \\\n",
              "0             0         0           0    0     0          0        0        0   \n",
              "1             0         0           0    0     0          0        0        0   \n",
              "2             0         0           0    0     0          0        0        0   \n",
              "3             0         0           0    0     0          0        0        1   \n",
              "4             0         0           0    0     1          1        0        0   \n",
              "5             0         0           0    0     1          0        0        0   \n",
              "6             0         0           0    0     0          0        0        0   \n",
              "7             0         0           0    0     0          0        0        0   \n",
              "8             0         0           0    1     0          0        0        0   \n",
              "9             1         0           0    0     0          0        0        1   \n",
              "10            0         0           1    0     0          0        1        0   \n",
              "11            0         0           0    0     0          0        0        0   \n",
              "12            0         0           0    0     0          0        0        0   \n",
              "13            0         0           0    0     0          0        0        0   \n",
              "14            0         1           0    0     0          0        0        0   \n",
              "15            0         0           0    0     0          0        0        0   \n",
              "16            2         0           0    0     0          0        0        0   \n",
              "\n",
              "    area  areas  ...  understand  update  utilization  view  walk  want  way  \\\n",
              "0      0      0  ...           0       0            0     0     0     1    0   \n",
              "1      0      0  ...           0       0            1     0     0     0    0   \n",
              "2      0      0  ...           0       0            0     0     0     0    0   \n",
              "3      0      0  ...           0       0            0     0     0     0    0   \n",
              "4      0      0  ...           0       0            0     0     0     1    0   \n",
              "5      0      0  ...           0       0            0     0     0     0    0   \n",
              "6      0      0  ...           0       1            0     0     0     0    0   \n",
              "7      0      0  ...           0       0            0     1     0     0    0   \n",
              "8      0      0  ...           0       0            1     0     0     0    0   \n",
              "9      0      0  ...           0       0            0     0     0     0    0   \n",
              "10     0      0  ...           1       0            0     0     0     0    1   \n",
              "11     1      1  ...           0       0            0     0     0     0    0   \n",
              "12     0      0  ...           0       0            0     0     1     0    0   \n",
              "13     0      0  ...           0       0            0     0     0     0    0   \n",
              "14     0      0  ...           0       0            0     0     0     0    0   \n",
              "15     0      0  ...           0       0            0     0     0     1    0   \n",
              "16     0      0  ...           0       1            0     0     0     0    0   \n",
              "\n",
              "    weakens  wfe  year  \n",
              "0         0    0     0  \n",
              "1         0    0     0  \n",
              "2         0    0     0  \n",
              "3         0    1     1  \n",
              "4         0    2     3  \n",
              "5         0    1     0  \n",
              "6         0    0     2  \n",
              "7         1    0     3  \n",
              "8         0    0     0  \n",
              "9         0    0     0  \n",
              "10        0    0     0  \n",
              "11        0    0     0  \n",
              "12        0    0     4  \n",
              "13        0    0     3  \n",
              "14        0    0     2  \n",
              "15        0    0     2  \n",
              "16        0    0     0  \n",
              "\n",
              "[17 rows x 299 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-34e1d9dc-cf21-46d3-8a8c-573742009a99\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>acquisition</th>\n",
              "      <th>actually</th>\n",
              "      <th>additional</th>\n",
              "      <th>ags</th>\n",
              "      <th>amat</th>\n",
              "      <th>analytics</th>\n",
              "      <th>answers</th>\n",
              "      <th>applied</th>\n",
              "      <th>area</th>\n",
              "      <th>areas</th>\n",
              "      <th>...</th>\n",
              "      <th>understand</th>\n",
              "      <th>update</th>\n",
              "      <th>utilization</th>\n",
              "      <th>view</th>\n",
              "      <th>walk</th>\n",
              "      <th>want</th>\n",
              "      <th>way</th>\n",
              "      <th>weakens</th>\n",
              "      <th>wfe</th>\n",
              "      <th>year</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>1</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>4</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>3</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>2</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>2</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>...</td>\n",
              "      <td>0</td>\n",
              "      <td>1</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "      <td>0</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>17 rows × 299 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-34e1d9dc-cf21-46d3-8a8c-573742009a99')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-34e1d9dc-cf21-46d3-8a8c-573742009a99 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-34e1d9dc-cf21-46d3-8a8c-573742009a99');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 6
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Conver to frequency–inverse document frequency matrix\n",
        "\n",
        "1. *In Term Frequency(TF)*, you just count the number of words occurred in each document. The main issue with this Term Frequency is that it will give more weight to longer documents. Term frequency is basically the output of the BoW model.\n",
        "\n",
        "2. *IDF(Inverse Document Frequency)* measures the amount of information a given word provides across the document. IDF is the logarithmically scaled inverse ratio of the number of documents that contain the word and the total number of documents.\n",
        "$$\n",
        "* *TF(w)*=*Frequency of w occurs in the document / the total number of words*\n",
        "\n",
        "* *IDF(w)*=*log(the number of documents containing the term)/ the total number of documents(+1)*\n",
        "\n",
        "\n",
        "$$\n",
        "W = log\\frac{documents}{W}\n",
        "$$\n",
        "\n",
        "* *documents*\t=\tNumber of documents\n",
        "\n",
        "* *W*\t=\tNumber of documents containing word\n",
        "\n",
        "$$TF-IDF(w)=TF(w)*IDF(w)$$\n",
        "\n",
        "3. TF-IDF(Term Frequency-Inverse Document Frequency) normalizes the document term matrix. It is the product of TF and IDF. Word with high tf-idf in a document, it is most of the times occurred in given documents and must be absent in the other documents. \n"
      ],
      "metadata": {
        "id": "CANUXXQffhhb"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# convert a collection of raw documents to a matrix of TF-IDF features\n",
        "\n",
        "# requency–inverse document frequency vector\n",
        "tfidf_vectorizer = TfidfVectorizer(**tf_vectorizer.get_params())\n",
        "\n",
        "# document term frequency–inverse document frequency\n",
        "dtm_tfidf = tfidf_vectorizer.fit_transform(label_df['text'])           # label_df['text'] performs slightly better than label_df['clean_text']\n",
        "pd.DataFrame(data = dtm_tfidf.toarray(), columns = tfidf_vectorizer.get_feature_names())"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 745
        },
        "id": "_RR4CkAJQwJP",
        "outputId": "f3a7fca1-081b-4977-f3f4-55768ca1451c"
      },
      "execution_count": 7,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/sklearn/feature_extraction/text.py:2032: UserWarning: Only (<class 'numpy.float64'>, <class 'numpy.float32'>, <class 'numpy.float16'>) 'dtype' should be used. <class 'numpy.int64'> 'dtype' will be converted to np.float64.\n",
            "  UserWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function get_feature_names is deprecated; get_feature_names is deprecated in 1.0 and will be removed in 1.2. Please use get_feature_names_out instead.\n",
            "  warnings.warn(msg, category=FutureWarning)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "    acquisition  actually  additional       ags      amat  analytics  \\\n",
              "0      0.000000   0.00000    0.000000  0.000000  0.000000   0.000000   \n",
              "1      0.000000   0.00000    0.000000  0.000000  0.000000   0.000000   \n",
              "2      0.000000   0.00000    0.000000  0.000000  0.000000   0.000000   \n",
              "3      0.000000   0.00000    0.000000  0.000000  0.000000   0.000000   \n",
              "4      0.000000   0.00000    0.000000  0.000000  0.143644   0.164507   \n",
              "5      0.000000   0.00000    0.000000  0.000000  0.206585   0.000000   \n",
              "6      0.000000   0.00000    0.000000  0.000000  0.000000   0.000000   \n",
              "7      0.000000   0.00000    0.000000  0.000000  0.000000   0.000000   \n",
              "8      0.000000   0.00000    0.000000  0.202825  0.000000   0.000000   \n",
              "9      0.150076   0.00000    0.000000  0.000000  0.000000   0.000000   \n",
              "10     0.000000   0.00000    0.116543  0.000000  0.000000   0.000000   \n",
              "11     0.000000   0.00000    0.000000  0.000000  0.000000   0.000000   \n",
              "12     0.000000   0.00000    0.000000  0.000000  0.000000   0.000000   \n",
              "13     0.000000   0.00000    0.000000  0.000000  0.000000   0.000000   \n",
              "14     0.000000   0.20302    0.000000  0.000000  0.000000   0.000000   \n",
              "15     0.000000   0.00000    0.000000  0.000000  0.000000   0.000000   \n",
              "16     0.372368   0.00000    0.000000  0.000000  0.000000   0.000000   \n",
              "\n",
              "     answers   applied      area     areas  ...  understand    update  \\\n",
              "0   0.000000  0.000000  0.000000  0.000000  ...    0.000000  0.000000   \n",
              "1   0.000000  0.000000  0.000000  0.000000  ...    0.000000  0.000000   \n",
              "2   0.000000  0.000000  0.000000  0.000000  ...    0.000000  0.000000   \n",
              "3   0.000000  0.153353  0.000000  0.000000  ...    0.000000  0.000000   \n",
              "4   0.000000  0.000000  0.000000  0.000000  ...    0.000000  0.000000   \n",
              "5   0.000000  0.000000  0.000000  0.000000  ...    0.000000  0.000000   \n",
              "6   0.000000  0.000000  0.000000  0.000000  ...    0.000000  0.190025   \n",
              "7   0.000000  0.000000  0.000000  0.000000  ...    0.000000  0.000000   \n",
              "8   0.000000  0.000000  0.000000  0.000000  ...    0.000000  0.000000   \n",
              "9   0.000000  0.150076  0.000000  0.000000  ...    0.000000  0.000000   \n",
              "10  0.116543  0.000000  0.000000  0.000000  ...    0.116543  0.000000   \n",
              "11  0.000000  0.000000  0.115487  0.115487  ...    0.000000  0.000000   \n",
              "12  0.000000  0.000000  0.000000  0.000000  ...    0.000000  0.000000   \n",
              "13  0.000000  0.000000  0.000000  0.000000  ...    0.000000  0.000000   \n",
              "14  0.000000  0.000000  0.000000  0.000000  ...    0.000000  0.000000   \n",
              "15  0.000000  0.000000  0.000000  0.000000  ...    0.000000  0.000000   \n",
              "16  0.000000  0.000000  0.000000  0.000000  ...    0.000000  0.186184   \n",
              "\n",
              "    utilization      view      walk      want       way   weakens       wfe  \\\n",
              "0      0.000000  0.000000  0.000000  0.167822  0.000000  0.000000  0.000000   \n",
              "1      0.188621  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
              "2      0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
              "3      0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.137551   \n",
              "4      0.000000  0.000000  0.000000  0.128842  0.000000  0.000000  0.257685   \n",
              "5      0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.185297   \n",
              "6      0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
              "7      0.000000  0.122644  0.000000  0.000000  0.000000  0.122644  0.000000   \n",
              "8      0.177103  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
              "9      0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
              "10     0.000000  0.000000  0.000000  0.000000  0.116543  0.000000  0.000000   \n",
              "11     0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
              "12     0.000000  0.000000  0.149272  0.000000  0.000000  0.000000  0.000000   \n",
              "13     0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
              "14     0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
              "15     0.000000  0.000000  0.000000  0.123848  0.000000  0.000000  0.000000   \n",
              "16     0.000000  0.000000  0.000000  0.000000  0.000000  0.000000  0.000000   \n",
              "\n",
              "        year  \n",
              "0   0.000000  \n",
              "1   0.000000  \n",
              "2   0.000000  \n",
              "3   0.093006  \n",
              "4   0.261353  \n",
              "5   0.000000  \n",
              "6   0.230493  \n",
              "7   0.194845  \n",
              "8   0.000000  \n",
              "9   0.000000  \n",
              "10  0.000000  \n",
              "11  0.000000  \n",
              "12  0.316199  \n",
              "13  0.341619  \n",
              "14  0.215026  \n",
              "15  0.167481  \n",
              "16  0.000000  \n",
              "\n",
              "[17 rows x 299 columns]"
            ],
            "text/html": [
              "\n",
              "  <div id=\"df-0ec13a6a-9348-45d4-8f3e-989823ce32d0\">\n",
              "    <div class=\"colab-df-container\">\n",
              "      <div>\n",
              "<style scoped>\n",
              "    .dataframe tbody tr th:only-of-type {\n",
              "        vertical-align: middle;\n",
              "    }\n",
              "\n",
              "    .dataframe tbody tr th {\n",
              "        vertical-align: top;\n",
              "    }\n",
              "\n",
              "    .dataframe thead th {\n",
              "        text-align: right;\n",
              "    }\n",
              "</style>\n",
              "<table border=\"1\" class=\"dataframe\">\n",
              "  <thead>\n",
              "    <tr style=\"text-align: right;\">\n",
              "      <th></th>\n",
              "      <th>acquisition</th>\n",
              "      <th>actually</th>\n",
              "      <th>additional</th>\n",
              "      <th>ags</th>\n",
              "      <th>amat</th>\n",
              "      <th>analytics</th>\n",
              "      <th>answers</th>\n",
              "      <th>applied</th>\n",
              "      <th>area</th>\n",
              "      <th>areas</th>\n",
              "      <th>...</th>\n",
              "      <th>understand</th>\n",
              "      <th>update</th>\n",
              "      <th>utilization</th>\n",
              "      <th>view</th>\n",
              "      <th>walk</th>\n",
              "      <th>want</th>\n",
              "      <th>way</th>\n",
              "      <th>weakens</th>\n",
              "      <th>wfe</th>\n",
              "      <th>year</th>\n",
              "    </tr>\n",
              "  </thead>\n",
              "  <tbody>\n",
              "    <tr>\n",
              "      <th>0</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.00000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.167822</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>1</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.00000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.188621</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>2</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.00000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>3</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.00000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.153353</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.137551</td>\n",
              "      <td>0.093006</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>4</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.00000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.143644</td>\n",
              "      <td>0.164507</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.128842</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.257685</td>\n",
              "      <td>0.261353</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>5</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.00000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.206585</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.185297</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>6</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.00000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.190025</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.230493</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>7</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.00000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.122644</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.122644</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.194845</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>8</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.00000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.202825</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.177103</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>9</th>\n",
              "      <td>0.150076</td>\n",
              "      <td>0.00000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.150076</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>10</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.00000</td>\n",
              "      <td>0.116543</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.116543</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.116543</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.116543</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>11</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.00000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.115487</td>\n",
              "      <td>0.115487</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>12</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.00000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.149272</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.316199</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>13</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.00000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.341619</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>14</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.20302</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.215026</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>15</th>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.00000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.123848</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.167481</td>\n",
              "    </tr>\n",
              "    <tr>\n",
              "      <th>16</th>\n",
              "      <td>0.372368</td>\n",
              "      <td>0.00000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>...</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.186184</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "      <td>0.000000</td>\n",
              "    </tr>\n",
              "  </tbody>\n",
              "</table>\n",
              "<p>17 rows × 299 columns</p>\n",
              "</div>\n",
              "      <button class=\"colab-df-convert\" onclick=\"convertToInteractive('df-0ec13a6a-9348-45d4-8f3e-989823ce32d0')\"\n",
              "              title=\"Convert this dataframe to an interactive table.\"\n",
              "              style=\"display:none;\">\n",
              "        \n",
              "  <svg xmlns=\"http://www.w3.org/2000/svg\" height=\"24px\"viewBox=\"0 0 24 24\"\n",
              "       width=\"24px\">\n",
              "    <path d=\"M0 0h24v24H0V0z\" fill=\"none\"/>\n",
              "    <path d=\"M18.56 5.44l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94zm-11 1L8.5 8.5l.94-2.06 2.06-.94-2.06-.94L8.5 2.5l-.94 2.06-2.06.94zm10 10l.94 2.06.94-2.06 2.06-.94-2.06-.94-.94-2.06-.94 2.06-2.06.94z\"/><path d=\"M17.41 7.96l-1.37-1.37c-.4-.4-.92-.59-1.43-.59-.52 0-1.04.2-1.43.59L10.3 9.45l-7.72 7.72c-.78.78-.78 2.05 0 2.83L4 21.41c.39.39.9.59 1.41.59.51 0 1.02-.2 1.41-.59l7.78-7.78 2.81-2.81c.8-.78.8-2.07 0-2.86zM5.41 20L4 18.59l7.72-7.72 1.47 1.35L5.41 20z\"/>\n",
              "  </svg>\n",
              "      </button>\n",
              "      \n",
              "  <style>\n",
              "    .colab-df-container {\n",
              "      display:flex;\n",
              "      flex-wrap:wrap;\n",
              "      gap: 12px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert {\n",
              "      background-color: #E8F0FE;\n",
              "      border: none;\n",
              "      border-radius: 50%;\n",
              "      cursor: pointer;\n",
              "      display: none;\n",
              "      fill: #1967D2;\n",
              "      height: 32px;\n",
              "      padding: 0 0 0 0;\n",
              "      width: 32px;\n",
              "    }\n",
              "\n",
              "    .colab-df-convert:hover {\n",
              "      background-color: #E2EBFA;\n",
              "      box-shadow: 0px 1px 2px rgba(60, 64, 67, 0.3), 0px 1px 3px 1px rgba(60, 64, 67, 0.15);\n",
              "      fill: #174EA6;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert {\n",
              "      background-color: #3B4455;\n",
              "      fill: #D2E3FC;\n",
              "    }\n",
              "\n",
              "    [theme=dark] .colab-df-convert:hover {\n",
              "      background-color: #434B5C;\n",
              "      box-shadow: 0px 1px 3px 1px rgba(0, 0, 0, 0.15);\n",
              "      filter: drop-shadow(0px 1px 2px rgba(0, 0, 0, 0.3));\n",
              "      fill: #FFFFFF;\n",
              "    }\n",
              "  </style>\n",
              "\n",
              "      <script>\n",
              "        const buttonEl =\n",
              "          document.querySelector('#df-0ec13a6a-9348-45d4-8f3e-989823ce32d0 button.colab-df-convert');\n",
              "        buttonEl.style.display =\n",
              "          google.colab.kernel.accessAllowed ? 'block' : 'none';\n",
              "\n",
              "        async function convertToInteractive(key) {\n",
              "          const element = document.querySelector('#df-0ec13a6a-9348-45d4-8f3e-989823ce32d0');\n",
              "          const dataTable =\n",
              "            await google.colab.kernel.invokeFunction('convertToInteractive',\n",
              "                                                     [key], {});\n",
              "          if (!dataTable) return;\n",
              "\n",
              "          const docLinkHtml = 'Like what you see? Visit the ' +\n",
              "            '<a target=\"_blank\" href=https://colab.research.google.com/notebooks/data_table.ipynb>data table notebook</a>'\n",
              "            + ' to learn more about interactive tables.';\n",
              "          element.innerHTML = '';\n",
              "          dataTable['output_type'] = 'display_data';\n",
              "          await google.colab.output.renderOutput(dataTable, element);\n",
              "          const docLink = document.createElement('div');\n",
              "          docLink.innerHTML = docLinkHtml;\n",
              "          element.appendChild(docLink);\n",
              "        }\n",
              "      </script>\n",
              "    </div>\n",
              "  </div>\n",
              "  "
            ]
          },
          "metadata": {},
          "execution_count": 7
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Fit Latent Dirichet Allocation models"
      ],
      "metadata": {
        "id": "A1GrZYKGgIkB"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "  n_components: number of topics which we want to mimic the situation which we don't know exactly how many topics are within the documents, default=10\n",
        "\"\"\"\n",
        "\n",
        "# for TF DTM\n",
        "lda_tf = LDA_model(n_components=10, random_state=0)\n",
        "lda_tf.fit(dtm_tf)\n",
        "\n",
        "# for TFIDF DTM\n",
        "lda_tfidf = LDA_model(n_components=10, random_state=0)\n",
        "lda_tfidf.fit(dtm_tfidf)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "CjtYYvRjVRZp",
        "outputId": "121f551c-d6b5-4b73-a1ff-5369a1088b52"
      },
      "execution_count": 8,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "LatentDirichletAllocation(random_state=0)"
            ]
          },
          "metadata": {},
          "execution_count": 8
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "\"\"\"\n",
        "  pyLDAvis is used to visualize modeled results:\n",
        "    - topic prevalance\n",
        "    - topic similarity\n",
        "    - topic interpretation  \n",
        "\n",
        "   lda_tf: lda object\n",
        "   dtm_tf: numerical representation\n",
        "   tf_vectorizer: contains all the word recall vocabulary for us to plot\n",
        "\n",
        "   1- each topic is represented by circle the area of each circle represents an important quantity which is \n",
        "      the topic prevalence, indices here indicate the sorted order by the area\n",
        "\n",
        "   2- the distance between two circles that represents the topic similarity but it is only an approximation\n",
        "      to the original topic similarity matrix because we are only using a two dimensional scatter plots\n",
        "      that reprsend original dimention\n",
        "\n",
        "   3- red bar term frequency for this particular topic, blue bar is the overall term frequency(appering\n",
        "      in other topics)\n",
        "\n",
        "   lambda: helps you to interpret the topic more clearly because it tries to strike a balance between choosing\n",
        "           only jargons choosing words that are more lament oriented terms \n",
        "\n",
        "\n",
        "\"\"\"\n",
        "pyLDAvis.sklearn.prepare(lda_tf, dtm_tf, tf_vectorizer, mds='tsne')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 1000
        },
        "id": "hiJKwsfhV8ku",
        "outputId": "d1822b4a-9574-4602-dcb8-d7155ec19d20"
      },
      "execution_count": 9,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stderr",
          "text": [
            "/usr/local/lib/python3.7/dist-packages/sklearn/utils/deprecation.py:87: FutureWarning: Function get_feature_names is deprecated; get_feature_names is deprecated in 1.0 and will be removed in 1.2. Please use get_feature_names_out instead.\n",
            "  warnings.warn(msg, category=FutureWarning)\n",
            "/usr/local/lib/python3.7/dist-packages/pyLDAvis/_prepare.py:247: FutureWarning: In a future version of pandas all arguments of DataFrame.drop except for the argument 'labels' will be keyword-only\n",
            "  by='saliency', ascending=False).head(R).drop('saliency', 1)\n",
            "/usr/local/lib/python3.7/dist-packages/sklearn/manifold/_t_sne.py:783: FutureWarning: The default initialization in TSNE will change from 'random' to 'pca' in 1.2.\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/sklearn/manifold/_t_sne.py:793: FutureWarning: The default learning rate in TSNE will change from 200.0 to 'auto' in 1.2.\n",
            "  FutureWarning,\n",
            "/usr/local/lib/python3.7/dist-packages/sklearn/manifold/_t_sne.py:827: FutureWarning: 'square_distances' has been introduced in 0.24 to help phase out legacy squaring behavior. The 'legacy' setting will be removed in 1.1 (renaming of 0.26), and the default setting will be changed to True. In 1.3, 'square_distances' will be removed altogether, and distances will be squared by default. Set 'square_distances'=True to silence this warning.\n",
            "  FutureWarning,\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "PreparedData(topic_coordinates=               x          y  topics  cluster       Freq\n",
              "topic                                                  \n",
              "7      32.420952   8.115230       1        1  34.914568\n",
              "8       0.648955  71.707100       2        1  18.704777\n",
              "6      71.019684   9.971006       3        1  12.280822\n",
              "2       3.785981  33.185135       4        1   9.088846\n",
              "0     -42.666500  -9.222070       5        1   9.074419\n",
              "4      -3.621689 -43.526516       6        1   8.230877\n",
              "3      -3.632402  -4.171708       7        1   6.870401\n",
              "1     -36.584400  35.804413       8        1   0.278430\n",
              "5      44.391380  50.615383       9        1   0.278430\n",
              "9      40.235809 -31.591665      10        1   0.278430, topic_info=            Term       Freq      Total Category  logprob  loglift\n",
              "298         year  15.000000  15.000000  Default  30.0000  30.0000\n",
              "160         nand   5.000000   5.000000  Default  29.0000  29.0000\n",
              "209      quarter   8.000000   8.000000  Default  28.0000  28.0000\n",
              "138         like   6.000000   6.000000  Default  27.0000  27.0000\n",
              "85       foundry   9.000000   9.000000  Default  26.0000  26.0000\n",
              "..           ...        ...        ...      ...      ...      ...\n",
              "138         like   0.005513   6.084250  Topic10  -5.7004  -1.1226\n",
              "30         china   0.005513   2.837966  Topic10  -5.7004  -0.3600\n",
              "19           bit   0.005513   3.561011  Topic10  -5.7004  -0.5870\n",
              "279     thinking   0.005513   2.000301  Topic10  -5.7004  -0.0102\n",
              "119  incremental   0.005513   3.508795  Topic10  -5.7004  -0.5722\n",
              "\n",
              "[499 rows x 6 columns], token_table=      Topic      Freq         Term\n",
              "term                              \n",
              "0         5  0.421040  acquisition\n",
              "0         6  0.421040  acquisition\n",
              "2         2  0.781261   additional\n",
              "3         3  0.829076          ags\n",
              "4         4  0.559471         amat\n",
              "...     ...       ...          ...\n",
              "298       1  0.578555         year\n",
              "298       2  0.064284         year\n",
              "298       3  0.064284         year\n",
              "298       4  0.128568         year\n",
              "298       7  0.128568         year\n",
              "\n",
              "[359 rows x 3 columns], R=30, lambda_step=0.01, plot_opts={'xlab': 'PC1', 'ylab': 'PC2'}, topic_order=[8, 9, 7, 3, 1, 5, 4, 2, 6, 10])"
            ],
            "text/html": [
              "\n",
              "<link rel=\"stylesheet\" type=\"text/css\" href=\"https://cdn.jsdelivr.net/gh/bmabey/pyLDAvis@3.3.1/pyLDAvis/js/ldavis.v1.0.0.css\">\n",
              "\n",
              "\n",
              "<div id=\"ldavis_el5318140481231148112453874178\"></div>\n",
              "<script type=\"text/javascript\">\n",
              "\n",
              "var ldavis_el5318140481231148112453874178_data = {\"mdsDat\": {\"x\": [32.42095184326172, 0.648955225944519, 71.01968383789062, 3.7859814167022705, -42.666500091552734, -3.6216893196105957, -3.632401704788208, -36.58440017700195, 44.391380310058594, 40.235809326171875], \"y\": [8.115229606628418, 71.70709991455078, 9.971006393432617, 33.18513488769531, -9.22206974029541, -43.52651596069336, -4.171707630157471, 35.804412841796875, 50.61538314819336, -31.591665267944336], \"topics\": [1, 2, 3, 4, 5, 6, 7, 8, 9, 10], \"cluster\": [1, 1, 1, 1, 1, 1, 1, 1, 1, 1], \"Freq\": [34.914568284833386, 18.70477738583353, 12.280821530672444, 9.08884626368352, 9.074419329985984, 8.230876809473248, 6.870400889942382, 0.2784298351918357, 0.2784298351918357, 0.2784298351918357]}, \"tinfo\": {\"Term\": [\"year\", \"nand\", \"quarter\", \"like\", \"foundry\", \"think\", \"business\", \"just\", \"point\", \"growth\", \"margins\", \"gross\", \"wfe\", \"new\", \"euv\", \"margin\", \"january\", \"guess\", \"assume\", \"rate\", \"expect\", \"low\", \"trends\", \"run\", \"talk\", \"display\", \"memory\", \"products\", \"seeing\", \"nanometer\", \"run\", \"trends\", \"low\", \"comment\", \"capital\", \"intensity\", \"calendar\", \"say\", \"sort\", \"aside\", \"etch\", \"conceptually\", \"sales\", \"terms\", \"rate\", \"seeing\", \"capex\", \"comments\", \"community\", \"concern\", \"current\", \"fourth\", \"half\", \"implied\", \"investment\", \"levels\", \"looking\", \"picks\", \"raise\", \"recent\", \"logic\", \"foundry\", \"just\", \"year\", \"growth\", \"point\", \"china\", \"model\", \"starting\", \"leadership\", \"gary\", \"does\", \"things\", \"relative\", \"think\", \"january\", \"guess\", \"memory\", \"like\", \"quarter\", \"going\", \"euv\", \"new\", \"nanometer\", \"nodes\", \"node\", \"expectations\", \"specific\", \"hoping\", \"market\", \"additional\", \"answers\", \"change\", \"critical\", \"different\", \"dimensions\", \"evolving\", \"exposed\", \"heard\", \"heavy\", \"image\", \"intel\", \"layers\", \"mind\", \"moves\", \"multiple\", \"patterning\", \"plus\", \"remain\", \"sense\", \"session\", \"product\", \"products\", \"tsmc\", \"kind\", \"expect\", \"lot\", \"bit\", \"guys\", \"come\", \"type\", \"outlook\", \"given\", \"relative\", \"color\", \"things\", \"assume\", \"quarterly\", \"strength\", \"logical\", \"peak\", \"asking\", \"illogical\", \"isn\", \"john\", \"necessarily\", \"sustained\", \"teens\", \"ags\", \"bringing\", \"capacity\", \"customer\", \"idle\", \"long\", \"october\", \"performance\", \"predictable\", \"rise\", \"starts\", \"subscription\", \"term\", \"transactional\", \"probably\", \"quarter\", \"revenue\", \"utilization\", \"business\", \"foundry\", \"january\", \"guess\", \"just\", \"year\", \"driven\", \"sustain\", \"better\", \"outlook\", \"going\", \"strong\", \"model\", \"starting\", \"want\", \"follow\", \"question\", \"come\", \"fiscal\", \"level\", \"nand\", \"amat\", \"jan\", \"numbers\", \"wfe\", \"analytics\", \"breakdown\", \"data\", \"expectation\", \"mentioned\", \"past\", \"semi\", \"spoken\", \"trying\", \"dram\", \"maybe\", \"business\", \"expect\", \"dan\", \"talking\", \"quick\", \"guidance\", \"think\", \"want\", \"guys\", \"follow\", \"year\", \"quarter\", \"kind\", \"growth\", \"logic\", \"foundry\", \"just\", \"customers\", \"declines\", \"focus\", \"productivity\", \"based\", \"big\", \"cost\", \"decelerating\", \"dramatically\", \"drivers\", \"help\", \"improve\", \"pretty\", \"systems\", \"team\", \"start\", \"base\", \"coming\", \"excellent\", \"installed\", \"particularly\", \"rates\", \"second\", \"service\", \"sneak\", \"trajectory\", \"quick\", \"doing\", \"higher\", \"batch\", \"memory\", \"growth\", \"thinking\", \"said\", \"bit\", \"applied\", \"talked\", \"utilization\", \"kokusai\", \"follow\", \"construction\", \"fabs\", \"flattish\", \"lcd\", \"oled\", \"opex\", \"profile\", \"pushing\", \"puts\", \"remainder\", \"resuming\", \"samsung\", \"takes\", \"complete\", \"exposure\", \"feel\", \"future\", \"gaining\", \"hurdles\", \"need\", \"overcome\", \"portfolio\", \"processing\", \"regulatory\", \"table\", \"technology\", \"acquisition\", \"talk\", \"display\", \"batch\", \"point\", \"update\", \"outlook\", \"china\", \"look\", \"hear\", \"follow\", \"obviously\", \"fiscal\", \"margin\", \"ssg\", \"basis\", \"corporate\", \"drop\", \"hardly\", \"highest\", \"lower\", \"million\", \"points\", \"segment\", \"surprising\", \"walk\", \"higher\", \"doing\", \"gross\", \"margins\", \"revenue\", \"like\", \"guidance\", \"question\", \"little\", \"lot\", \"incremental\", \"level\", \"year\", \"january\", \"guess\", \"just\", \"complete\", \"driven\", \"basis\", \"corporate\", \"drop\", \"hardly\", \"highest\", \"lower\", \"million\", \"points\", \"segment\", \"surprising\", \"walk\", \"complete\", \"exposure\", \"feel\", \"future\", \"gaining\", \"hurdles\", \"need\", \"overcome\", \"portfolio\", \"processing\", \"regulatory\", \"table\", \"technology\", \"construction\", \"fabs\", \"flattish\", \"lcd\", \"oled\", \"opex\", \"driven\", \"break\", \"equal\", \"outperform\", \"perform\", \"putting\", \"semiconductor\", \"memory\", \"outlook\", \"just\", \"follow\", \"guys\", \"year\", \"kind\", \"update\", \"type\", \"talk\", \"think\", \"logic\", \"relative\", \"probably\", \"talking\", \"little\", \"come\", \"does\", \"like\", \"china\", \"bit\", \"thinking\", \"incremental\", \"basis\", \"corporate\", \"drop\", \"hardly\", \"highest\", \"lower\", \"million\", \"points\", \"segment\", \"surprising\", \"walk\", \"complete\", \"exposure\", \"feel\", \"future\", \"gaining\", \"hurdles\", \"need\", \"overcome\", \"portfolio\", \"processing\", \"regulatory\", \"table\", \"technology\", \"construction\", \"fabs\", \"flattish\", \"lcd\", \"oled\", \"opex\", \"driven\", \"break\", \"equal\", \"outperform\", \"perform\", \"putting\", \"semiconductor\", \"memory\", \"outlook\", \"just\", \"follow\", \"guys\", \"year\", \"kind\", \"update\", \"type\", \"talk\", \"think\", \"logic\", \"relative\", \"probably\", \"talking\", \"little\", \"come\", \"does\", \"like\", \"china\", \"bit\", \"thinking\", \"incremental\", \"basis\", \"corporate\", \"drop\", \"hardly\", \"highest\", \"lower\", \"million\", \"points\", \"segment\", \"surprising\", \"walk\", \"complete\", \"exposure\", \"feel\", \"future\", \"gaining\", \"hurdles\", \"need\", \"overcome\", \"portfolio\", \"processing\", \"regulatory\", \"table\", \"technology\", \"construction\", \"fabs\", \"flattish\", \"lcd\", \"oled\", \"opex\", \"driven\", \"break\", \"equal\", \"outperform\", \"perform\", \"putting\", \"semiconductor\", \"memory\", \"outlook\", \"just\", \"follow\", \"guys\", \"year\", \"kind\", \"update\", \"type\", \"talk\", \"think\", \"logic\", \"relative\", \"probably\", \"talking\", \"little\", \"come\", \"does\", \"like\", \"china\", \"bit\", \"thinking\", \"incremental\"], \"Freq\": [15.0, 5.0, 8.0, 6.0, 9.0, 7.0, 4.0, 11.0, 4.0, 5.0, 3.0, 3.0, 3.0, 3.0, 3.0, 2.0, 5.0, 5.0, 2.0, 4.0, 3.0, 3.0, 3.0, 3.0, 2.0, 2.0, 5.0, 4.0, 4.0, 2.0, 3.527563133891798, 3.527563133732831, 3.5275631335256947, 2.667181881658592, 2.6671818815662083, 2.6671818815662083, 2.667181881355357, 2.667181881286993, 1.8068006292745615, 1.8068006291754057, 1.8068006291754057, 1.8068006287258884, 1.8068006287258884, 1.8068006287258884, 3.5275637000294946, 3.5275637797877275, 0.9464193767799494, 0.9464193767799494, 0.9464193767799494, 0.9464193767799494, 0.9464193767799494, 0.9464193767799494, 0.9464193767799494, 0.9464193767799494, 0.9464193767799494, 0.9464193767799494, 0.9464193767799494, 0.9464193767799494, 0.9464193767799494, 0.9464193767799494, 5.790064037295554, 5.790058038094191, 6.6892389073789165, 8.689845218357284, 3.527560747138689, 2.6671784789229815, 1.8067987981588818, 1.8067999824174399, 1.8067999824174399, 1.8068014170524669, 1.8068010807846913, 1.8068010054358246, 1.8068005399187894, 1.8068002300747534, 3.527560731768195, 2.667178098140502, 2.667180735920938, 2.667180061856445, 2.6671783477606117, 3.01382226924307, 1.806800984830426, 3.1994541519076427, 3.1994541517380397, 2.4190994806110044, 2.4190994806110044, 1.6387448092999668, 1.6387448091165564, 1.6387448091063814, 1.6387448090146763, 1.6387448090146763, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 0.8583901379204549, 1.6387448447081847, 2.4191011115653964, 1.6387462306622005, 2.419104388376151, 1.6387440223584673, 1.6387461917384658, 1.638746396931334, 1.6387487192697672, 1.6387463276650343, 0.8583915355804516, 0.8583906687867965, 0.8583905279058556, 0.8583904989553449, 0.858390364659426, 0.8583902187709703, 2.1902579607016186, 1.4837231345703619, 1.4837231345250017, 0.7771883083558678, 0.7771883083558678, 0.7771883083558677, 0.7771883083558677, 0.7771883083558677, 0.7771883083558677, 0.7771883083558677, 0.7771883083558677, 0.7771883083558677, 0.7771883082432821, 0.7771883082432821, 0.7771883082432821, 0.7771883082432821, 0.7771883082432821, 0.7771883082432821, 0.7771883082432821, 0.7771883082432821, 0.7771883082432821, 0.7771883082432821, 0.7771883082432821, 0.7771883082432821, 0.7771883082432821, 0.7771883082432821, 1.483728130534295, 4.309874046727908, 0.7771879039565238, 0.7771882169699104, 1.4837233151282838, 2.190260716754366, 1.4837253547075282, 1.4837254635629538, 1.4837255411569308, 1.4837206817712785, 0.7771935335044979, 0.7771895978865823, 0.7771895349288899, 0.7771894125139115, 0.7771888882343225, 0.7771888500920253, 0.7771888390531051, 0.7771888390531051, 0.7771888389220003, 0.7771887525006134, 0.7771886357073182, 0.7771885686535401, 0.7771883761342356, 0.7771883148410994, 3.9277145063222676, 1.3521600821694746, 1.3521600821694746, 1.3521600821694746, 1.9960519902616327, 0.7082760255018373, 0.7082760255018373, 0.7082760255018373, 0.7082760255018373, 0.7082760255018373, 0.7082760255018373, 0.7082760255018373, 0.7082760255018373, 0.7082760255018373, 1.352160829454562, 1.3521648749478354, 1.9960516130474981, 1.3521607313972361, 0.7082762724610102, 0.7082782086431018, 0.7082755439862752, 0.7082760866408513, 1.9960597176779507, 0.7082772365127206, 0.7082781772195523, 0.7082776041269797, 1.996056876461519, 1.0927359539806196, 0.7082767467122779, 0.7082757929533254, 0.9467448321610884, 0.9467421551561278, 0.9177103648682009, 1.3446147988225854, 1.3446147988225854, 1.3446147988225854, 1.3446147988225854, 0.7043220372502185, 0.7043220372502185, 0.7043220372502185, 0.7043220372502185, 0.7043220372502185, 0.7043220372502185, 0.7043220372502185, 0.7043220372502185, 0.7043220372502185, 0.7043220372502185, 0.7043220372502185, 0.7043220371498262, 0.7043220371498263, 0.7043220371498263, 0.7043220371498263, 0.7043220371498263, 0.7043220371498263, 0.7043220371498263, 0.7043220371498263, 0.7043220371498263, 0.7043220371498263, 0.7043220371498263, 1.3446152776194837, 0.7043216646116298, 0.7043214674997453, 0.7043221113730352, 1.3446212936264008, 1.3446168061783537, 0.7043234885417097, 0.7043234353782133, 0.704322816493229, 0.7043225908562428, 0.7043224944409899, 0.704322119929149, 0.7043221113730352, 0.7043221086077133, 0.6793340018093864, 0.6793340018093864, 0.6793340018093864, 0.6793340018093864, 0.6793340018093864, 0.6793340018093864, 0.6793340018093864, 0.6793340018093864, 0.6793340018093864, 0.6793340018093864, 0.6793340018093864, 0.6793340018093864, 0.6793340018093864, 0.679334001791195, 0.679334001791195, 0.679334001791195, 0.679334001791195, 0.679334001791195, 0.679334001791195, 0.679334001791195, 0.679334001791195, 0.679334001791195, 0.679334001791195, 0.679334001791195, 0.679334001791195, 0.679334001791195, 1.2969114161301016, 1.2969160838153297, 1.2969144493752256, 0.6793339303612795, 1.2969128097247846, 0.6793361296357482, 0.6793359093504151, 0.6793353155703402, 0.6793348639490584, 0.6793346998233029, 0.6793346084159307, 0.6793345273913947, 0.679334147214006, 1.8037975937303912, 1.221927402139093, 0.6400572105171803, 0.6400572105171803, 0.6400572105171803, 0.6400572105171803, 0.6400572105171803, 0.6400572105171803, 0.6400572105171803, 0.6400572105171803, 0.6400572105171803, 0.6400572105171803, 0.6400572105171803, 0.6400577282468819, 0.6400575492032843, 1.2219308768060109, 1.2219308768060109, 0.6400575436042386, 1.8038068022063618, 0.6400576764726932, 0.6400585757375601, 0.6400594056176372, 0.6400581161076427, 0.640058909741371, 0.6400583964788247, 2.385681727267314, 0.6400579406607881, 0.6400577105850963, 1.2219339315127102, 0.058187019195523056, 0.058187019497996206, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724489893053, 0.005512724489893053, 0.005512724489893053, 0.005512724489893053, 0.005512724489893053, 0.005512724489893053, 0.005512724557870135, 0.005512724549672724, 0.005512724549672724, 0.005512724549672724, 0.005512724549672724, 0.005512724549672724, 0.005512724549672724, 0.005512724524586983, 0.005512724523762024, 0.005512724521621557, 0.005512724520439028, 0.005512724519591514, 0.005512724516359571, 0.005512724516339605, 0.0055127245159305146, 0.00551272451459638, 0.005512724514023293, 0.005512724513913644, 0.005512724513430141, 0.0055127245128745025, 0.005512724512668397, 0.005512724512578354, 0.00551272451198967, 0.005512724511462448, 0.005512724511030034, 0.005512724510931095, 0.005512724510598296, 0.005512724510505972, 0.005512724509961735, 0.005512724509688695, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724489893053, 0.005512724489893053, 0.005512724489893053, 0.005512724489893053, 0.005512724489893053, 0.005512724489893053, 0.005512724557870135, 0.005512724549672724, 0.005512724549672724, 0.005512724549672724, 0.005512724549672724, 0.005512724549672724, 0.005512724549672724, 0.005512724524586983, 0.005512724523762024, 0.005512724521621557, 0.005512724520439028, 0.005512724519591514, 0.005512724516359571, 0.005512724516339605, 0.0055127245159305146, 0.00551272451459638, 0.005512724514023293, 0.005512724513913644, 0.005512724513430141, 0.0055127245128745025, 0.005512724512668397, 0.005512724512578354, 0.00551272451198967, 0.005512724511462448, 0.005512724511030034, 0.005512724510931095, 0.005512724510598296, 0.005512724510505972, 0.005512724509961735, 0.005512724509688695, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.0055127244862646614, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724490234605, 0.005512724489893053, 0.005512724489893053, 0.005512724489893053, 0.005512724489893053, 0.005512724489893053, 0.005512724489893053, 0.005512724557870135, 0.005512724549672724, 0.005512724549672724, 0.005512724549672724, 0.005512724549672724, 0.005512724549672724, 0.005512724549672724, 0.005512724524586983, 0.005512724523762024, 0.005512724521621557, 0.005512724520439028, 0.005512724519591514, 0.005512724516359571, 0.005512724516339605, 0.0055127245159305146, 0.00551272451459638, 0.005512724514023293, 0.005512724513913644, 0.005512724513430141, 0.0055127245128745025, 0.005512724512668397, 0.005512724512578354, 0.00551272451198967, 0.005512724511462448, 0.005512724511030034, 0.005512724510931095, 0.005512724510598296, 0.005512724510505972, 0.005512724509961735, 0.005512724509688695], \"Total\": [15.0, 5.0, 8.0, 6.0, 9.0, 7.0, 4.0, 11.0, 4.0, 5.0, 3.0, 3.0, 3.0, 3.0, 3.0, 2.0, 5.0, 5.0, 2.0, 4.0, 3.0, 3.0, 3.0, 3.0, 2.0, 2.0, 5.0, 4.0, 4.0, 2.0, 3.9411529188099133, 3.9411529187049843, 3.9411529185683145, 3.0807716665402842, 3.0807716664792815, 3.0807716664792815, 3.0807716663401705, 3.080771666295042, 2.2203904141711144, 2.2203904141056388, 2.2203904141056388, 2.2203904138090267, 2.2203904138090267, 2.2203904138090267, 4.58144582490064, 4.647687860220217, 1.36000916172896, 1.36000916172896, 1.36000916172896, 1.36000916172896, 1.36000916172896, 1.36000916172896, 1.36000916172896, 1.36000916172896, 1.36000916172896, 1.36000916172896, 1.36000916172896, 1.36000916172896, 1.36000916172896, 1.36000916172896, 8.572894046785965, 9.205608482857064, 11.313309721507471, 15.556001079316713, 5.865625125634323, 4.315923437223021, 2.8379662624835533, 2.926925124071117, 2.926925124071117, 3.0007451580539453, 3.000745126965649, 3.000745119513474, 3.000745076680668, 3.0007450473132273, 7.293460512608478, 5.07571067686492, 5.69328781471092, 5.848245183200189, 6.084249662297838, 8.694979842843937, 3.707279883388367, 3.621046594822898, 3.621046594715502, 2.840691923530902, 2.840691923530902, 2.0603372522297834, 2.06033725211366, 2.0603372521071863, 2.060337252049125, 2.060337252049125, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 1.2799825808852845, 2.6779136248740043, 4.5614542602972605, 2.9207183582859693, 5.82291754522893, 3.3481127956790875, 3.502588120319386, 3.561010834400851, 3.5646046747968994, 3.6272531172017404, 2.140363688577216, 3.464473978772847, 2.140363792710764, 3.0007450473132273, 2.140363809017592, 3.000745076680668, 2.619232388130729, 1.9126975619969173, 1.9126975619696733, 1.2061627358131246, 1.2061627358131246, 1.2061627358131246, 1.2061627358131246, 1.2061627358131246, 1.2061627358131246, 1.2061627358131246, 1.2061627358131246, 1.2061627358131246, 1.2061627357455047, 1.2061627357455047, 1.2061627357455047, 1.2061627357455047, 1.2061627357455047, 1.2061627357455047, 1.2061627357455047, 1.2061627357455047, 1.2061627357455047, 1.2061627357455047, 1.2061627357455047, 1.2061627357455047, 1.2061627357455047, 1.2061627357455047, 2.7730777251735748, 8.694979842843937, 1.7880328559214962, 1.8464554884029336, 4.624714141430385, 9.205608482857064, 5.07571067686492, 5.69328781471092, 11.313309721507471, 15.556001079316713, 2.0665064764889403, 2.0665437064173457, 2.0665437200341117, 3.464473978772847, 3.707279883388367, 2.066543869583295, 2.926925124071117, 2.926925124071117, 2.630404390414763, 3.9682995199112567, 2.6484134543250524, 3.6272531172017404, 2.464031863471515, 3.5087948623313423, 5.003245421459097, 1.787403591077533, 1.787403591077533, 1.787403591077533, 3.2116444281438903, 1.1435152059209621, 1.1435152059209621, 1.1435152059209621, 1.1435152059209621, 1.1435152059209621, 1.1435152059209621, 1.1435152059209621, 1.1435152059209621, 1.1435152059209621, 2.4276963565214214, 2.4939371768319893, 4.624714141430385, 3.3481127956790875, 1.7610915814487176, 2.003895722936413, 2.4241007260002565, 2.43192011752331, 7.293460512608478, 2.630404390414763, 3.5646046747968994, 3.9682995199112567, 15.556001079316713, 8.694979842843937, 5.82291754522893, 5.865625125634323, 8.572894046785965, 9.205608482857064, 11.313309721507471, 1.7802134327258436, 1.7802134327258436, 1.7802134327258436, 1.7802134327258436, 1.1399206711871916, 1.1399206711871916, 1.1399206711871916, 1.1399206711871916, 1.1399206711871916, 1.1399206711871916, 1.1399206711871916, 1.1399206711871916, 1.1399206711871916, 1.1399206711871916, 1.1399206711871916, 1.1399206711309111, 1.1399206711309113, 1.1399206711309113, 1.1399206711309113, 1.1399206711309113, 1.1399206711309113, 1.1399206711309113, 1.1399206711309113, 1.1399206711309113, 1.1399206711309113, 1.1399206711309113, 2.4241007260002565, 1.7217908286502257, 1.7217908106179225, 1.7574970392435358, 5.848245183200189, 5.865625125634323, 2.0003014236210093, 2.000301441924428, 3.561010834400851, 1.920275220925206, 1.9202752419374804, 1.8464554884029336, 1.7574970392435358, 3.9682995199112567, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 2.375073363764397, 2.5951596447378518, 2.595160287684204, 1.7574970392435358, 4.315923437223021, 1.9775846898212472, 3.464473978772847, 2.8379662624835533, 1.8975587187801792, 1.977585252753965, 3.9682995199112567, 1.8975588076448524, 2.464031863471515, 2.245238484565218, 1.6633682929797673, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.7217908106179225, 1.7217908286502257, 3.384129133765769, 3.384129133765769, 1.7880328559214962, 6.084249662297838, 2.43192011752331, 2.6484134543250524, 2.7222331097957606, 3.502588120319386, 3.5087946374638994, 3.5087948623313423, 15.556001079316713, 5.07571067686492, 5.69328781471092, 11.313309721507471, 1.1172042753407736, 2.0665064764889403, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 2.0665064764889403, 1.3599727884953334, 1.3599727884953334, 1.3599727884953334, 1.3599727884953334, 1.3599727884953334, 1.3599727884953334, 5.848245183200189, 3.464473978772847, 11.313309721507471, 3.9682995199112567, 3.5646046747968994, 15.556001079316713, 5.82291754522893, 1.9775846898212472, 2.140363688577216, 2.5951596447378518, 7.293460512608478, 8.572894046785965, 3.0007450473132273, 2.7730777251735748, 2.003895722936413, 2.7222331097957606, 3.6272531172017404, 3.000745119513474, 6.084249662297838, 2.8379662624835533, 3.561010834400851, 2.0003014236210093, 3.5087946374638994, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 2.0665064764889403, 1.3599727884953334, 1.3599727884953334, 1.3599727884953334, 1.3599727884953334, 1.3599727884953334, 1.3599727884953334, 5.848245183200189, 3.464473978772847, 11.313309721507471, 3.9682995199112567, 3.5646046747968994, 15.556001079316713, 5.82291754522893, 1.9775846898212472, 2.140363688577216, 2.5951596447378518, 7.293460512608478, 8.572894046785965, 3.0007450473132273, 2.7730777251735748, 2.003895722936413, 2.7222331097957606, 3.6272531172017404, 3.000745119513474, 6.084249662297838, 2.8379662624835533, 3.561010834400851, 2.0003014236210093, 3.5087946374638994, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.0814981013784983, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753407736, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 1.1172042753506637, 2.0665064764889403, 1.3599727884953334, 1.3599727884953334, 1.3599727884953334, 1.3599727884953334, 1.3599727884953334, 1.3599727884953334, 5.848245183200189, 3.464473978772847, 11.313309721507471, 3.9682995199112567, 3.5646046747968994, 15.556001079316713, 5.82291754522893, 1.9775846898212472, 2.140363688577216, 2.5951596447378518, 7.293460512608478, 8.572894046785965, 3.0007450473132273, 2.7730777251735748, 2.003895722936413, 2.7222331097957606, 3.6272531172017404, 3.000745119513474, 6.084249662297838, 2.8379662624835533, 3.561010834400851, 2.0003014236210093, 3.5087946374638994], \"Category\": [\"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Default\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic1\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic2\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic3\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic4\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic5\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic6\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic7\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic8\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic9\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\", \"Topic10\"], \"logprob\": [30.0, 29.0, 28.0, 27.0, 26.0, 25.0, 24.0, 23.0, 22.0, 21.0, 20.0, 19.0, 18.0, 17.0, 16.0, 15.0, 14.0, 13.0, 12.0, 11.0, 10.0, 9.0, 8.0, 7.0, 6.0, 5.0, 4.0, 3.0, 2.0, 1.0, -4.0706, -4.0706, -4.0706, -4.3502, -4.3502, -4.3502, -4.3502, -4.3502, -4.7397, -4.7397, -4.7397, -4.7397, -4.7397, -4.7397, -4.0706, -4.0706, -5.3863, -5.3863, -5.3863, -5.3863, -5.3863, -5.3863, -5.3863, -5.3863, -5.3863, -5.3863, -5.3863, -5.3863, -5.3863, -5.3863, -3.5751, -3.5751, -3.4307, -3.1691, -4.0706, -4.3502, -4.7397, -4.7397, -4.7397, -4.7397, -4.7397, -4.7397, -4.7397, -4.7397, -4.0706, -4.3502, -4.3502, -4.3502, -4.3502, -4.228, -4.7397, -3.5441, -3.5441, -3.8237, -3.8237, -4.2132, -4.2132, -4.2132, -4.2132, -4.2132, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.2132, -3.8237, -4.2132, -3.8237, -4.2132, -4.2132, -4.2132, -4.2132, -4.2132, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -4.8598, -3.5024, -3.8918, -3.8918, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -3.8918, -2.8255, -4.5384, -4.5384, -3.8918, -3.5024, -3.8918, -3.8918, -3.8918, -3.8918, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -4.5384, -2.6173, -3.6837, -3.6837, -3.6837, -3.2942, -4.3303, -4.3303, -4.3303, -4.3303, -4.3303, -4.3303, -4.3303, -4.3303, -4.3303, -3.6837, -3.6837, -3.2942, -3.6837, -4.3303, -4.3303, -4.3303, -4.3303, -3.2942, -4.3303, -4.3303, -4.3303, -3.2942, -3.8967, -4.3303, -4.3303, -4.0401, -4.0401, -4.0713, -3.6877, -3.6877, -3.6877, -3.6877, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -3.6877, -4.3343, -4.3343, -4.3343, -3.6877, -3.6877, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.3343, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -3.6262, -3.6262, -3.6262, -4.2729, -3.6262, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -4.2729, -3.1157, -3.5051, -4.1518, -4.1518, -4.1518, -4.1518, -4.1518, -4.1518, -4.1518, -4.1518, -4.1518, -4.1518, -4.1518, -4.1518, -4.1518, -3.5051, -3.5051, -4.1518, -3.1157, -4.1518, -4.1518, -4.1518, -4.1518, -4.1518, -4.1518, -2.8361, -4.1518, -4.1518, -3.5051, -6.5497, -6.5497, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004, -5.7004], \"loglift\": [30.0, 29.0, 28.0, 27.0, 26.0, 25.0, 24.0, 23.0, 22.0, 21.0, 20.0, 19.0, 18.0, 17.0, 16.0, 15.0, 14.0, 13.0, 12.0, 11.0, 10.0, 9.0, 8.0, 7.0, 6.0, 5.0, 4.0, 3.0, 2.0, 1.0, 0.9414, 0.9414, 0.9414, 0.9081, 0.9081, 0.9081, 0.9081, 0.9081, 0.8461, 0.8461, 0.8461, 0.8461, 0.8461, 0.8461, 0.7909, 0.7765, 0.6897, 0.6897, 0.6897, 0.6897, 0.6897, 0.6897, 0.6897, 0.6897, 0.6897, 0.6897, 0.6897, 0.6897, 0.6897, 0.6897, 0.6598, 0.5886, 0.5268, 0.47, 0.5438, 0.571, 0.6007, 0.5699, 0.5699, 0.545, 0.545, 0.545, 0.545, 0.545, 0.3259, 0.4088, 0.294, 0.2671, 0.2276, -0.0073, 0.3335, 1.5526, 1.5526, 1.5157, 1.5157, 1.4475, 1.4475, 1.4475, 1.4475, 1.4475, 1.2768, 1.2768, 1.2768, 1.2768, 1.2768, 1.2768, 1.2768, 1.2768, 1.2768, 1.2768, 1.2768, 1.2768, 1.2768, 1.2768, 1.2768, 1.2768, 1.2768, 1.2768, 1.2768, 1.2768, 1.2768, 1.1853, 1.0421, 1.0985, 0.798, 0.9619, 0.9168, 0.9003, 0.8993, 0.8818, 0.7627, 0.2811, 0.7627, 0.4248, 0.7627, 0.4248, 1.9183, 1.8432, 1.8432, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.6576, 1.4717, 1.3953, 1.2639, 1.2318, 0.9603, 0.6613, 0.8672, 0.7524, 0.0657, -0.2528, 1.1192, 1.1192, 1.1192, 0.6025, 0.5348, 1.1192, 0.7711, 0.7711, 0.8779, 0.4667, 0.8711, 0.5566, 0.9433, 0.5898, 2.1561, 2.1191, 2.1191, 2.1191, 1.9225, 1.9191, 1.9191, 1.9191, 1.9191, 1.9191, 1.9191, 1.9191, 1.9191, 1.9191, 1.8129, 1.786, 1.5579, 1.4914, 1.4873, 1.3581, 1.1677, 1.1645, 1.1023, 1.0861, 0.7822, 0.6749, 0.3448, 0.3241, 0.2914, 0.2841, 0.1948, 0.1236, -0.1137, 2.1191, 2.1191, 2.1191, 2.1191, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.9182, 1.8104, 1.5058, 1.5058, 1.4853, 0.9297, 0.9267, 1.3559, 1.3559, 0.7791, 1.3967, 1.3967, 1.4359, 1.4853, 0.6709, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.9998, 1.8922, 1.8036, 1.8036, 1.5467, 1.295, 1.4288, 0.8681, 1.0675, 1.4701, 1.4288, 0.7323, 1.4701, 1.2088, 2.459, 2.3695, 2.1534, 2.1534, 2.1534, 2.1534, 2.1534, 2.1534, 2.1534, 2.1534, 2.1534, 2.1534, 2.1534, 1.6884, 1.6884, 1.6593, 1.6593, 1.6506, 1.4621, 1.3431, 1.2578, 1.2303, 0.9782, 0.9765, 0.9765, 0.803, 0.6073, 0.4925, 0.4524, -0.277, -0.892, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, -0.0428, 0.3756, 0.3756, 0.3756, 0.3756, 0.3756, 0.3756, -1.0831, -0.5595, -1.7429, -0.6953, -0.588, -2.0614, -1.0787, 0.0012, -0.0779, -0.2706, -1.3039, -1.4655, -0.4158, -0.3369, -0.012, -0.3184, -0.6054, -0.4158, -1.1226, -0.36, -0.587, -0.0102, -0.5722, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, -0.0428, 0.3756, 0.3756, 0.3756, 0.3756, 0.3756, 0.3756, -1.0831, -0.5595, -1.7429, -0.6953, -0.588, -2.0614, -1.0787, 0.0012, -0.0779, -0.2706, -1.3039, -1.4655, -0.4158, -0.3369, -0.012, -0.3184, -0.6054, -0.4158, -1.1226, -0.36, -0.587, -0.0102, -0.5722, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.6047, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, 0.5722, -0.0428, 0.3756, 0.3756, 0.3756, 0.3756, 0.3756, 0.3756, -1.0831, -0.5595, -1.7429, -0.6953, -0.588, -2.0614, -1.0787, 0.0012, -0.0779, -0.2706, -1.3039, -1.4655, -0.4158, -0.3369, -0.012, -0.3184, -0.6054, -0.4158, -1.1226, -0.36, -0.587, -0.0102, -0.5722]}, \"token.table\": {\"Topic\": [5, 6, 2, 3, 4, 4, 2, 2, 5, 1, 3, 3, 5, 5, 7, 5, 6, 1, 3, 5, 1, 2, 5, 1, 4, 3, 2, 3, 4, 1, 3, 1, 1, 2, 1, 6, 1, 2, 1, 2, 3, 5, 1, 1, 1, 6, 1, 1, 6, 7, 5, 2, 1, 3, 5, 4, 6, 4, 5, 5, 2, 2, 1, 6, 1, 2, 5, 7, 4, 5, 5, 1, 3, 5, 7, 1, 1, 2, 2, 5, 2, 4, 4, 2, 2, 6, 6, 6, 3, 5, 6, 6, 5, 1, 3, 4, 5, 6, 1, 3, 4, 1, 6, 6, 1, 2, 1, 2, 1, 2, 3, 1, 7, 1, 4, 5, 1, 3, 6, 7, 3, 4, 7, 1, 2, 4, 1, 7, 1, 6, 2, 2, 5, 5, 7, 7, 2, 6, 3, 3, 2, 1, 5, 1, 3, 7, 5, 2, 1, 1, 3, 4, 1, 3, 7, 3, 1, 2, 3, 4, 7, 1, 2, 4, 6, 5, 6, 2, 6, 1, 2, 1, 3, 7, 1, 1, 5, 6, 7, 1, 2, 7, 1, 2, 3, 4, 3, 3, 2, 6, 1, 1, 2, 7, 1, 7, 7, 1, 7, 2, 3, 4, 1, 2, 3, 5, 4, 7, 2, 1, 3, 2, 2, 4, 5, 2, 3, 6, 2, 2, 2, 4, 2, 6, 3, 6, 6, 1, 2, 3, 6, 1, 6, 5, 4, 2, 3, 1, 3, 1, 2, 1, 6, 7, 6, 3, 5, 1, 3, 6, 2, 6, 5, 1, 2, 6, 6, 6, 1, 1, 3, 4, 3, 1, 3, 7, 4, 5, 1, 1, 5, 5, 1, 6, 1, 2, 2, 6, 6, 3, 7, 3, 1, 1, 5, 1, 6, 1, 5, 1, 3, 7, 4, 1, 2, 5, 2, 5, 1, 2, 4, 7, 5, 1, 3, 3, 3, 1, 3, 3, 7, 1, 3, 3, 5, 6, 6, 1, 6, 2, 5, 1, 4, 5, 6, 3, 3, 1, 1, 2, 1, 2, 4, 5, 1, 5, 5, 3, 1, 4, 1, 2, 1, 2, 1, 6, 3, 5, 7, 2, 3, 4, 2, 4, 1, 2, 3, 4, 7], \"Freq\": [0.4210396256623584, 0.4210396256623584, 0.7812606319285705, 0.8290755222029972, 0.5594707345290449, 0.8744964604074695, 0.7812606319285705, 0.5207586855795551, 0.5207586855795551, 0.9007424943354339, 0.8290755221565176, 0.7635824942693773, 0.8772540276929126, 0.8772540276496007, 0.9246433245933402, 0.5689910012197923, 0.5689910012197923, 0.4838997550864752, 0.4838997550864752, 0.8772540276496007, 0.28081913998676516, 0.5616382799735303, 0.28081913998676516, 0.7353088300438677, 0.8744964604074695, 0.8290755222029972, 0.21622958077376617, 0.21622958077376617, 0.43245916154753233, 0.9737820016904648, 0.8290755222029972, 0.7352891643235068, 0.9737820016464941, 0.7812606319285705, 0.7047300126287497, 0.35236500631437484, 0.4672102919078001, 0.4672102919078001, 0.2756907135202779, 0.5513814270405558, 0.2756907135202779, 0.8772540276929126, 0.9737820016272121, 0.7352891643235068, 0.7352891643235068, 0.895091454689409, 0.90074249445576, 0.7352891643235068, 0.8950914546814851, 0.9246433245933402, 0.8772540276496007, 0.7812606319285705, 0.7352891643235068, 0.8290755222029972, 0.5617303979494249, 0.5678296407375789, 0.5678296407375789, 0.8744964604074695, 0.8772540276496007, 0.5617303979494249, 0.7812606319285705, 0.7812606319285705, 0.38533265353422613, 0.38533265353422613, 0.6665011256685041, 0.33325056283425203, 0.5807906415577415, 0.5807906415577415, 0.4119131279798402, 0.4119131279798402, 0.8772540276496007, 0.48390847615393473, 0.48390847615393473, 0.8772540276496007, 0.9246433245933402, 0.7353088300438677, 0.9007424943354339, 0.8284897532909894, 0.7812606319285705, 0.8772540276929126, 0.5973514400653118, 0.2986757200326559, 0.8744964604074695, 0.970714866193988, 0.7812606319285705, 0.895091454689409, 0.8950914546814851, 0.895091454689409, 0.40583890769623576, 0.40583890769623576, 0.40583890769623576, 0.8950914546814851, 0.5617303979494249, 0.2519971073207607, 0.2519971073207607, 0.2519971073207607, 0.2519971073207607, 0.2519971073207607, 0.6517765785035681, 0.21725885950118937, 0.10862942975059468, 0.7352891643235068, 0.895091454689409, 0.895091454689409, 0.6665011240132874, 0.3332505620066437, 0.4672102954673435, 0.4672102954673435, 0.5394790959705063, 0.2697395479852531, 0.2697395479852531, 0.5909939960755732, 0.2954969980377866, 0.6819392501779477, 0.1704848125444869, 0.1704848125444869, 0.5269362972039253, 0.17564543240130845, 0.17564543240130845, 0.17564543240130845, 0.41119771689639595, 0.41119771689639595, 0.41119771689639595, 0.2805360176600725, 0.561072035320145, 0.2805360176600725, 0.7352891643235068, 0.9246433245933402, 0.5056672012533519, 0.5056672012533519, 0.7812606319285705, 0.7812606319285705, 0.8772540276496007, 0.5807906476403579, 0.5807906476403579, 0.9246433245933402, 0.970714866224393, 0.895091454689409, 0.8290755222029972, 0.8290755221565176, 0.7812606319285705, 0.7352891643235068, 0.8772540276496007, 0.5699963111678624, 0.2849981555839312, 0.2849981555839312, 0.8772540276929126, 0.7812606319285705, 0.9737820016464941, 0.7352891643235068, 0.8290755221565176, 0.5594707345290449, 0.5910502372946501, 0.19701674576488337, 0.19701674576488337, 0.8290755221565176, 0.6187402424502232, 0.08839146320717475, 0.08839146320717475, 0.08839146320717475, 0.08839146320717475, 0.34347043118251286, 0.34347043118251286, 0.17173521559125643, 0.17173521559125643, 0.5689910012197923, 0.5689910012197923, 0.7812606319285705, 0.8950914546814851, 0.6665011171082078, 0.3332505585541039, 0.5699962746386216, 0.2849981373193108, 0.2849981373193108, 0.7352891643235068, 0.49307641311796374, 0.16435880437265457, 0.16435880437265457, 0.32871760874530914, 0.36734546957112957, 0.36734546957112957, 0.36734546957112957, 0.699880339971009, 0.1166467233285015, 0.1166467233285015, 0.1166467233285015, 0.8290755221565176, 0.8290755222029972, 0.5269929146871601, 0.5269929146871601, 0.7352891643235068, 0.28550316670086073, 0.5710063334017215, 0.28550316670086073, 1.0149314382485475, 0.9246433245933402, 0.8907739706712237, 0.5909939960755732, 0.2954969980377866, 0.970714866224393, 0.40097240992665456, 0.40097240992665456, 0.5129743890727894, 0.17099146302426313, 0.17099146302426313, 0.17099146302426313, 0.8744964604074695, 0.9246433245933402, 0.7812606319285705, 0.6833109544046556, 0.3416554772023278, 0.7812606319285705, 0.7812606319285705, 0.799481069396248, 0.199870267349062, 0.7040538199278065, 0.8290755221565176, 0.895091454689409, 0.8284897533155614, 0.970714866139277, 0.7040538199278065, 0.5594707345290449, 0.5269928900075281, 0.5269928900075281, 0.8290755222029972, 0.8950914546814851, 0.8950914546814851, 0.2886441076270431, 0.2886441076270431, 0.2886441076270431, 0.2886441076270431, 0.7353088300438677, 0.895091454689409, 0.8772540276929126, 0.8744964604074695, 0.7812606319285705, 0.8290755221565176, 0.7353088300438677, 0.8290755222029972, 0.7352891643235068, 0.7812606319285705, 0.6951003750729831, 0.23170012502432766, 0.9246433245933402, 0.895091454689409, 0.8290755222029972, 0.8772540276496007, 0.36061015921845724, 0.36061015921845724, 0.895091454689409, 0.7468500781439878, 0.3734250390719939, 0.5617303979494249, 0.4384566600629827, 0.4384566600629827, 0.8950914546814851, 0.8950914546814851, 0.8950914546814851, 0.7353088300438677, 0.3450266767977654, 0.4600355690636872, 0.1150088922659218, 0.5228218092963783, 0.3775845491069104, 0.3775845491069104, 0.3775845491069104, 0.4125241122508926, 0.4125241122508926, 0.7352891643235068, 0.8730868273634448, 0.2182717068408612, 0.8772540276929126, 0.7352891643235068, 0.895091454689409, 0.6665011417050366, 0.3332505708525183, 0.7812606319285705, 0.8950914546814851, 0.8950914546814851, 0.5592738392296663, 0.5592738392296663, 0.8290755222029972, 1.0149314381863306, 0.4999246508755855, 0.4999246508755855, 0.90074249445576, 0.8950914546814851, 0.9737820017047292, 0.8772540276929126, 0.8606429950333351, 0.21516074875833377, 0.9246433245933402, 0.8744964604074695, 0.7353088300438677, 0.7812606319285705, 0.8772540276929126, 0.7812606319285705, 0.8772540276929126, 0.9007424943088724, 0.9707148661970378, 0.8744964604074695, 0.6011897691091577, 0.8772540276929127, 0.6833109544046556, 0.3416554772023278, 0.8290755222029972, 0.5228218093038253, 0.48389972006819454, 0.48389972006819454, 0.8290755222029972, 0.9246433245933402, 0.4838997582749631, 0.4838997582749631, 0.8290755221565176, 0.8772540276496007, 0.895091454689409, 0.8950914546814851, 0.38533274899973036, 0.38533274899973036, 0.5207586798812447, 0.5207586798812447, 0.49902796265997706, 0.49902796265997706, 0.8772540276496007, 0.895091454689409, 0.8290755221565176, 0.8290755222029972, 0.90074249445576, 0.666501135182179, 0.3332505675910895, 0.548436505974777, 0.13710912649369425, 0.2742182529873885, 0.13710912649369425, 0.4999246554500612, 0.4999246554500612, 0.8772540276929126, 0.8290755222029972, 1.014931438213352, 0.8744964604074695, 0.34238152308079867, 0.6847630461615973, 0.4672103181981841, 0.4672103181981841, 0.5056673451949051, 0.5056673451949051, 0.5415781784509391, 0.5415781784509391, 0.9246433245933402, 0.3801696817584461, 0.3801696817584461, 0.3801696817584461, 0.31136697177213085, 0.6227339435442617, 0.5785548582897964, 0.0642838731433107, 0.0642838731433107, 0.1285677462866214, 0.1285677462866214], \"Term\": [\"acquisition\", \"acquisition\", \"additional\", \"ags\", \"amat\", \"analytics\", \"answers\", \"applied\", \"applied\", \"aside\", \"asking\", \"assume\", \"base\", \"based\", \"basis\", \"batch\", \"batch\", \"better\", \"better\", \"big\", \"bit\", \"bit\", \"bit\", \"break\", \"breakdown\", \"bringing\", \"business\", \"business\", \"business\", \"calendar\", \"capacity\", \"capex\", \"capital\", \"change\", \"china\", \"china\", \"color\", \"color\", \"come\", \"come\", \"come\", \"coming\", \"comment\", \"comments\", \"community\", \"complete\", \"conceptually\", \"concern\", \"construction\", \"corporate\", \"cost\", \"critical\", \"current\", \"customer\", \"customers\", \"dan\", \"dan\", \"data\", \"decelerating\", \"declines\", \"different\", \"dimensions\", \"display\", \"display\", \"does\", \"does\", \"doing\", \"doing\", \"dram\", \"dram\", \"dramatically\", \"driven\", \"driven\", \"drivers\", \"drop\", \"equal\", \"etch\", \"euv\", \"evolving\", \"excellent\", \"expect\", \"expect\", \"expectation\", \"expectations\", \"exposed\", \"exposure\", \"fabs\", \"feel\", \"fiscal\", \"fiscal\", \"fiscal\", \"flattish\", \"focus\", \"follow\", \"follow\", \"follow\", \"follow\", \"follow\", \"foundry\", \"foundry\", \"foundry\", \"fourth\", \"future\", \"gaining\", \"gary\", \"gary\", \"given\", \"given\", \"going\", \"going\", \"going\", \"gross\", \"gross\", \"growth\", \"growth\", \"growth\", \"guess\", \"guess\", \"guess\", \"guess\", \"guidance\", \"guidance\", \"guidance\", \"guys\", \"guys\", \"guys\", \"half\", \"hardly\", \"hear\", \"hear\", \"heard\", \"heavy\", \"help\", \"higher\", \"higher\", \"highest\", \"hoping\", \"hurdles\", \"idle\", \"illogical\", \"image\", \"implied\", \"improve\", \"incremental\", \"incremental\", \"incremental\", \"installed\", \"intel\", \"intensity\", \"investment\", \"isn\", \"jan\", \"january\", \"january\", \"january\", \"john\", \"just\", \"just\", \"just\", \"just\", \"just\", \"kind\", \"kind\", \"kind\", \"kind\", \"kokusai\", \"kokusai\", \"layers\", \"lcd\", \"leadership\", \"leadership\", \"level\", \"level\", \"level\", \"levels\", \"like\", \"like\", \"like\", \"like\", \"little\", \"little\", \"little\", \"logic\", \"logic\", \"logic\", \"logic\", \"logical\", \"long\", \"look\", \"look\", \"looking\", \"lot\", \"lot\", \"lot\", \"low\", \"lower\", \"margin\", \"margins\", \"margins\", \"market\", \"maybe\", \"maybe\", \"memory\", \"memory\", \"memory\", \"memory\", \"mentioned\", \"million\", \"mind\", \"model\", \"model\", \"moves\", \"multiple\", \"nand\", \"nand\", \"nanometer\", \"necessarily\", \"need\", \"new\", \"node\", \"nodes\", \"numbers\", \"obviously\", \"obviously\", \"october\", \"oled\", \"opex\", \"outlook\", \"outlook\", \"outlook\", \"outlook\", \"outperform\", \"overcome\", \"particularly\", \"past\", \"patterning\", \"peak\", \"perform\", \"performance\", \"picks\", \"plus\", \"point\", \"point\", \"points\", \"portfolio\", \"predictable\", \"pretty\", \"probably\", \"probably\", \"processing\", \"product\", \"product\", \"productivity\", \"products\", \"products\", \"profile\", \"pushing\", \"puts\", \"putting\", \"quarter\", \"quarter\", \"quarter\", \"quarterly\", \"question\", \"question\", \"question\", \"quick\", \"quick\", \"raise\", \"rate\", \"rate\", \"rates\", \"recent\", \"regulatory\", \"relative\", \"relative\", \"remain\", \"remainder\", \"resuming\", \"revenue\", \"revenue\", \"rise\", \"run\", \"said\", \"said\", \"sales\", \"samsung\", \"say\", \"second\", \"seeing\", \"seeing\", \"segment\", \"semi\", \"semiconductor\", \"sense\", \"service\", \"session\", \"sneak\", \"sort\", \"specific\", \"spoken\", \"ssg\", \"start\", \"starting\", \"starting\", \"starts\", \"strength\", \"strong\", \"strong\", \"subscription\", \"surprising\", \"sustain\", \"sustain\", \"sustained\", \"systems\", \"table\", \"takes\", \"talk\", \"talk\", \"talked\", \"talked\", \"talking\", \"talking\", \"team\", \"technology\", \"teens\", \"term\", \"terms\", \"things\", \"things\", \"think\", \"think\", \"think\", \"think\", \"thinking\", \"thinking\", \"trajectory\", \"transactional\", \"trends\", \"trying\", \"tsmc\", \"tsmc\", \"type\", \"type\", \"update\", \"update\", \"utilization\", \"utilization\", \"walk\", \"want\", \"want\", \"want\", \"wfe\", \"wfe\", \"year\", \"year\", \"year\", \"year\", \"year\"]}, \"R\": 30, \"lambda.step\": 0.01, \"plot.opts\": {\"xlab\": \"PC1\", \"ylab\": \"PC2\"}, \"topic.order\": [8, 9, 7, 3, 1, 5, 4, 2, 6, 10]};\n",
              "\n",
              "function LDAvis_load_lib(url, callback){\n",
              "  var s = document.createElement('script');\n",
              "  s.src = url;\n",
              "  s.async = true;\n",
              "  s.onreadystatechange = s.onload = callback;\n",
              "  s.onerror = function(){console.warn(\"failed to load library \" + url);};\n",
              "  document.getElementsByTagName(\"head\")[0].appendChild(s);\n",
              "}\n",
              "\n",
              "if(typeof(LDAvis) !== \"undefined\"){\n",
              "   // already loaded: just create the visualization\n",
              "   !function(LDAvis){\n",
              "       new LDAvis(\"#\" + \"ldavis_el5318140481231148112453874178\", ldavis_el5318140481231148112453874178_data);\n",
              "   }(LDAvis);\n",
              "}else if(typeof define === \"function\" && define.amd){\n",
              "   // require.js is available: use it to load d3/LDAvis\n",
              "   require.config({paths: {d3: \"https://d3js.org/d3.v5\"}});\n",
              "   require([\"d3\"], function(d3){\n",
              "      window.d3 = d3;\n",
              "      LDAvis_load_lib(\"https://cdn.jsdelivr.net/gh/bmabey/pyLDAvis@3.3.1/pyLDAvis/js/ldavis.v3.0.0.js\", function(){\n",
              "        new LDAvis(\"#\" + \"ldavis_el5318140481231148112453874178\", ldavis_el5318140481231148112453874178_data);\n",
              "      });\n",
              "    });\n",
              "}else{\n",
              "    // require.js not available: dynamically load d3 & LDAvis\n",
              "    LDAvis_load_lib(\"https://d3js.org/d3.v5.js\", function(){\n",
              "         LDAvis_load_lib(\"https://cdn.jsdelivr.net/gh/bmabey/pyLDAvis@3.3.1/pyLDAvis/js/ldavis.v3.0.0.js\", function(){\n",
              "                 new LDAvis(\"#\" + \"ldavis_el5318140481231148112453874178\", ldavis_el5318140481231148112453874178_data);\n",
              "            })\n",
              "         });\n",
              "}\n",
              "</script>"
            ]
          },
          "metadata": {},
          "execution_count": 9
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#Doc2Vec & Embedding"
      ],
      "metadata": {
        "id": "ZOordboaVe1M"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "##**Doc2vec**\n",
        "The straightforward approach of averaging each of a text's words' word-vectors creates a quick and crude document-vector that can often be useful.\n",
        "\n",
        "**The basic idea is**: act as if a document has another floating word-like vector, which contributes to all training predictions, and is updated like other word-vectors, but we will call it a doc-vector.\n",
        "\n",
        "Gensim's Doc2Vec class implements this algorithm."
      ],
      "metadata": {
        "id": "p4jrAS1tfJGS"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "### Why we use Doc2Vec instead of Word2Vec?\n",
        "In text analysis, we usually get information from whole documents, not from single words.\n",
        "For example, the word \"apple\" does not provide any useful information for us. However, \"The apples in Netto taste delicious.\" means something.\n",
        "\n",
        "Therefore, we used Doc2Vec approach in our example, since we use Question and Answer labels in similar formats."
      ],
      "metadata": {
        "id": "NJlZQ_cmAeQt"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "sentences = [text.split() for text in label_df['clean_text']]\n",
        "len(sentences)\n",
        "\n",
        "tagged_documents = []\n",
        "\n",
        "for i, doc in enumerate(sentences):\n",
        "  tagged_documents.append(gensim.models.doc2vec.TaggedDocument(doc, [i]))\n",
        "\n",
        "\n",
        "d2v = Doc2Vec(vector_size=60, epochs=30)\n",
        "d2v.build_vocab(tagged_documents)\n",
        "\n",
        "d2v.train(tagged_documents, total_examples=d2v.corpus_count, epochs=d2v.epochs)\n",
        "\n",
        "doc_vecs = [d2v.infer_vector(tag_doc.words) for tag_doc in tagged_documents]\n",
        "\n",
        "# check dimension of doc_vecs before feeding it into the model\n",
        "print(f'dimension of doc_vecs: ({len(doc_vecs)},{len(doc_vecs[0])})')"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "Uhx0tEU-fK9u",
        "outputId": "0bc0c2d0-cebe-448b-8b77-e5ecc97034e4"
      },
      "execution_count": 10,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "dimension of doc_vecs: (17,60)\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Embedding"
      ],
      "metadata": {
        "id": "LiP_PTSOSB_4"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "In Doc2Vec, first we create a one-hot matrix by using the words in the document, and with input vector we create our matrix that represents the document in **numeric format**.\n",
        "First, we need a data structure that will work like a dictionary. There will be the words we have and their numerical equivalents next to them. \n",
        "\n",
        "For this porpuse, we used **Tokenizer**. Two more additional id numbers are necessary. These are:\n",
        "\n",
        "-0 = *null values in the matrix because of zero padding*\n",
        "\n",
        "-1 = *Out of vocubulary words, oov_token*"
      ],
      "metadata": {
        "id": "S_SOFLOUBFQF"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def tokenizer(text_data:pd.DataFrame, oov_token:str) -> np.ndarray:\n",
        "  \"\"\"\n",
        "  Convert text into token ids, later text to sequences and finally apply padding sequences to get the same length\n",
        "\n",
        "  Arg:\n",
        "    text_data(pd.DataFrame): corpus\n",
        "    oov_token(str): it will be added to word_index and used to replace out-of-vocabulary words during text_to_sequence calls\n",
        "\n",
        "  Return:\n",
        "    embedded matrix\n",
        "  \n",
        "  \"\"\"\n",
        "  my_tokenizer = tf.keras.preprocessing.text.Tokenizer(oov_token=oov_token)\n",
        "  my_tokenizer.fit_on_texts(label_df['clean_text'])\n",
        "  # number of lexicon + 1('NAN')\n",
        "  print(f'len of tokenizer: {len(my_tokenizer.word_index.keys())}')\n",
        "\n",
        "\n",
        "  txt_sequences = my_tokenizer.texts_to_sequences(label_df['clean_text'])\n",
        "  print(f'len of txt sequences {len(txt_sequences)}')\n",
        "\n",
        "\n",
        "  # better to put zeros in the beginning if you use rnn\n",
        "  pad_txt_sequences = tf.keras.preprocessing.sequence.pad_sequences(txt_sequences, maxlen=60)\n",
        "  print(f'len of pad txt sequences: {pad_txt_sequences.shape}')\n",
        "  \"\"\"\n",
        "    We synchronized the dimensions of the documents by zero-padding just above. \n",
        "    However, because some sentences were too long compared to others, many documents were filled with so many 0s. \n",
        "    We set maxlen to get rid of these very long 0 sequences. \n",
        "    Considering the length of the text sequences, we decided that maxlen = 60 is a balanced number.\n",
        "  \n",
        "  \"\"\"\n",
        "\n",
        "\n",
        "  return my_tokenizer, pad_txt_sequences\n",
        "\n",
        "\n",
        "my_tokenizer, pad_txt_sequences = tokenizer(label_df['clean_text'], 'NAN')\n",
        "pad_txt_sequences"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "rUM5VTzrcKJP",
        "outputId": "827fbc5f-4abb-46da-96c7-bd49dcb917cf"
      },
      "execution_count": 11,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "len of tokenizer: 312\n",
            "len of txt sequences 17\n",
            "len of pad txt sequences: (17, 60)\n"
          ]
        },
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "array([[  0,   0,   0, ...,  22,  20,  70],\n",
              "       [  0,   0,   0, ...,   8, 130,  77],\n",
              "       [  0,   0,   0, ..., 144,  42, 145],\n",
              "       ...,\n",
              "       [  0,   0,   0, ...,   2,   8, 285],\n",
              "       [  0,   0,   0, ..., 106,   3,   6],\n",
              "       [  0,   0,   0, ..., 310, 311, 312]], dtype=int32)"
            ]
          },
          "metadata": {},
          "execution_count": 11
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "# Modeling"
      ],
      "metadata": {
        "id": "2sXYhVQmxFiM"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Generate evaluation metrics and model checkpoint"
      ],
      "metadata": {
        "id": "EUMti4G9tlbE"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "def recall_m(y_true, y_pred):\n",
        "    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))\n",
        "    possible_positives = K.sum(K.round(K.clip(y_true, 0, 1)))\n",
        "    recall = true_positives / (possible_positives + K.epsilon())\n",
        "    return recall\n",
        "\n",
        "def precision_m(y_true, y_pred):\n",
        "    true_positives = K.sum(K.round(K.clip(y_true * y_pred, 0, 1)))\n",
        "    predicted_positives = K.sum(K.round(K.clip(y_pred, 0, 1)))\n",
        "    precision = true_positives / (predicted_positives + K.epsilon())\n",
        "    return precision\n",
        "\n",
        "def f1_m(y_true, y_pred):\n",
        "    precision = precision_m(y_true, y_pred)\n",
        "    recall = recall_m(y_true, y_pred)\n",
        "    return 2*((precision*recall)/(precision+recall+K.epsilon()))\n",
        "\n",
        "\n",
        "\"\"\"\n",
        "  callbacks are the special utilities or functions that are executed during training at given stages of the training procedure. \n",
        "  Callbacks can help you prevent overfitting, visualize training progress, debug your code, save checkpoints, generate logs, create a TensorBoard, etc. \n",
        "\"\"\"\n",
        "\n",
        "checkpoint_name = 'trained_models/Weights-e{epoch:03d}--v_loss_{val_loss:.5f}.hdf5'\n",
        "checkpoint = ModelCheckpoint(checkpoint_name, monitor='val_loss', verbose = 1, mode ='min')\n",
        "callbacks_list = [checkpoint]"
      ],
      "metadata": {
        "id": "iQcRgktNpfBt"
      },
      "execution_count": 12,
      "outputs": []
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Train test split"
      ],
      "metadata": {
        "id": "Gu_IksoXrQ7B"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# convert list data into np array\n",
        "target = np.array(label_df['level'], dtype=np.float32)\n",
        "doc_vecs = np.array(doc_vecs, dtype=np.float32)\n",
        "\n",
        "\"\"\"\n",
        "The train-test split is a technique for evaluating the performance of a machine learning algorithm.\n",
        "\n",
        "It can be used for classification or regression problems and can be used for any supervised learning algorithm.\n",
        "\n",
        "The procedure involves taking a dataset and dividing it into two subsets. The first subset is used to fit the model and is referred to as the training dataset. \n",
        "The second subset is not used to train the model; instead, the input element of the dataset is provided to the model, then predictions are made and compared to the expected values. \n",
        "This second dataset is referred to as the test dataset.\n",
        "\n",
        "  - Train Dataset: Used to fit the machine learning model.\n",
        "  - Test Dataset: Used to evaluate the fit machine learning model.\n",
        "\"\"\"\n",
        "\n",
        "# pad_txt_sequences is represented with pts\n",
        "x_train_pts, x_test_pts, y_train_pts, y_test_pts = train_test_split(pad_txt_sequences,target, test_size=0.2, random_state=42, shuffle=True)\n",
        "\n",
        "# doc_vecs is represented with dv\n",
        "x_train_dv, x_test_dv, y_train_dv, y_test_dv = train_test_split(doc_vecs, target, test_size=0.2, random_state=42, shuffle=True)\n",
        "\n",
        "# let's check if the data separated from the same index before we evaluate doc2vec and embeddings performances\n",
        "y_train_pts, y_train_dv, y_test_pts, y_test_dv"
      ],
      "metadata": {
        "id": "pi6J8MNJADOL",
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "outputId": "0fbd0f2a-fc1e-46fc-fc3a-9833fbd193a0"
      },
      "execution_count": 13,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([1., 1., 1., 1., 0., 0., 0., 0., 1., 0., 0., 1., 0.], dtype=float32),\n",
              " array([1., 1., 1., 1., 0., 0., 0., 0., 1., 0., 0., 1., 0.], dtype=float32),\n",
              " array([1., 0., 0., 1.], dtype=float32),\n",
              " array([1., 0., 0., 1.], dtype=float32))"
            ]
          },
          "metadata": {},
          "execution_count": 13
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Architecture"
      ],
      "metadata": {
        "id": "BFCTjaTut_UP"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "pts_model = tf.keras.Sequential([\n",
        "                                tf.keras.layers.Embedding(input_dim=(len(my_tokenizer.word_index.keys()) + 1), \n",
        "                                                          output_dim=150, input_length=60), # Combining the embedding layer\n",
        "                                tf.keras.layers.GRU(10, return_sequences=True, activation='relu'), # Usage of Recurrent layer\n",
        "                                tf.keras.layers.GRU(5, activation='relu'), \n",
        "                                tf.keras.layers.Dense(1,activation='sigmoid') # Lastly, we add another layer in here with activation function sigmoid\n",
        "])\n",
        "\n",
        "pts_model.summary()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "gOPz3GpTXYaO",
        "outputId": "e6505103-9044-481c-c5d5-a4b2a78900f2"
      },
      "execution_count": 14,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Model: \"sequential\"\n",
            "_________________________________________________________________\n",
            " Layer (type)                Output Shape              Param #   \n",
            "=================================================================\n",
            " embedding (Embedding)       (None, 60, 150)           46950     \n",
            "                                                                 \n",
            " gru (GRU)                   (None, 60, 10)            4860      \n",
            "                                                                 \n",
            " gru_1 (GRU)                 (None, 5)                 255       \n",
            "                                                                 \n",
            " dense (Dense)               (None, 1)                 6         \n",
            "                                                                 \n",
            "=================================================================\n",
            "Total params: 52,071\n",
            "Trainable params: 52,071\n",
            "Non-trainable params: 0\n",
            "_________________________________________________________________\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "pts_model.compile(loss='bce', metrics=['accuracy', f1_m,precision_m, recall_m])\n",
        "\n",
        "# for pad_txt_sequences input\n",
        "pts_history = pts_model.fit(x_train_pts, y_train_pts, epochs=20, validation_split=0.1, shuffle=True, callbacks = callbacks_list)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "T9ObK99YYKC3",
        "outputId": "a6ef4ee0-9a32-4c33-8e97-38eeb683ad53"
      },
      "execution_count": 15,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6936 - accuracy: 0.5455 - f1_m: 0.4444 - precision_m: 0.5000 - recall_m: 0.4000\n",
            "Epoch 1: saving model to trained_models/Weights-e001--v_loss_0.69651.hdf5\n",
            "1/1 [==============================] - 9s 9s/step - loss: 0.6936 - accuracy: 0.5455 - f1_m: 0.4444 - precision_m: 0.5000 - recall_m: 0.4000 - val_loss: 0.6965 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 2/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6785 - accuracy: 0.9091 - f1_m: 0.8889 - precision_m: 1.0000 - recall_m: 0.8000\n",
            "Epoch 2: saving model to trained_models/Weights-e002--v_loss_0.69930.hdf5\n",
            "1/1 [==============================] - 0s 181ms/step - loss: 0.6785 - accuracy: 0.9091 - f1_m: 0.8889 - precision_m: 1.0000 - recall_m: 0.8000 - val_loss: 0.6993 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 3/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6670 - accuracy: 0.9091 - f1_m: 0.8889 - precision_m: 1.0000 - recall_m: 0.8000\n",
            "Epoch 3: saving model to trained_models/Weights-e003--v_loss_0.70097.hdf5\n",
            "1/1 [==============================] - 0s 179ms/step - loss: 0.6670 - accuracy: 0.9091 - f1_m: 0.8889 - precision_m: 1.0000 - recall_m: 0.8000 - val_loss: 0.7010 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 4/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6571 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000\n",
            "Epoch 4: saving model to trained_models/Weights-e004--v_loss_0.70283.hdf5\n",
            "1/1 [==============================] - 0s 197ms/step - loss: 0.6571 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 0.7028 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 5/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6472 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000\n",
            "Epoch 5: saving model to trained_models/Weights-e005--v_loss_0.70425.hdf5\n",
            "1/1 [==============================] - 0s 197ms/step - loss: 0.6472 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 0.7042 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 6/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6373 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000\n",
            "Epoch 6: saving model to trained_models/Weights-e006--v_loss_0.70561.hdf5\n",
            "1/1 [==============================] - 0s 211ms/step - loss: 0.6373 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 0.7056 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 7/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6274 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000\n",
            "Epoch 7: saving model to trained_models/Weights-e007--v_loss_0.70719.hdf5\n",
            "1/1 [==============================] - 0s 250ms/step - loss: 0.6274 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 0.7072 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 8/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6173 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000\n",
            "Epoch 8: saving model to trained_models/Weights-e008--v_loss_0.70869.hdf5\n",
            "1/1 [==============================] - 0s 158ms/step - loss: 0.6173 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 0.7087 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 9/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6068 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000\n",
            "Epoch 9: saving model to trained_models/Weights-e009--v_loss_0.70996.hdf5\n",
            "1/1 [==============================] - 0s 164ms/step - loss: 0.6068 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 0.7100 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 10/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.5957 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000\n",
            "Epoch 10: saving model to trained_models/Weights-e010--v_loss_0.71165.hdf5\n",
            "1/1 [==============================] - 0s 155ms/step - loss: 0.5957 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 0.7116 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 11/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.5843 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000\n",
            "Epoch 11: saving model to trained_models/Weights-e011--v_loss_0.71286.hdf5\n",
            "1/1 [==============================] - 0s 153ms/step - loss: 0.5843 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 0.7129 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 12/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.5726 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000\n",
            "Epoch 12: saving model to trained_models/Weights-e012--v_loss_0.71495.hdf5\n",
            "1/1 [==============================] - 0s 170ms/step - loss: 0.5726 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 0.7150 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 13/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.5604 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000\n",
            "Epoch 13: saving model to trained_models/Weights-e013--v_loss_0.71647.hdf5\n",
            "1/1 [==============================] - 0s 260ms/step - loss: 0.5604 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 0.7165 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 14/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.5473 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000\n",
            "Epoch 14: saving model to trained_models/Weights-e014--v_loss_0.71798.hdf5\n",
            "1/1 [==============================] - 0s 168ms/step - loss: 0.5473 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 0.7180 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 15/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.5334 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000\n",
            "Epoch 15: saving model to trained_models/Weights-e015--v_loss_0.71985.hdf5\n",
            "1/1 [==============================] - 0s 192ms/step - loss: 0.5334 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 0.7198 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 16/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.5192 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000\n",
            "Epoch 16: saving model to trained_models/Weights-e016--v_loss_0.72161.hdf5\n",
            "1/1 [==============================] - 0s 151ms/step - loss: 0.5192 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 0.7216 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 17/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.5044 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000\n",
            "Epoch 17: saving model to trained_models/Weights-e017--v_loss_0.72328.hdf5\n",
            "1/1 [==============================] - 0s 253ms/step - loss: 0.5044 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 0.7233 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 18/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.4894 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000\n",
            "Epoch 18: saving model to trained_models/Weights-e018--v_loss_0.72524.hdf5\n",
            "1/1 [==============================] - 0s 242ms/step - loss: 0.4894 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 0.7252 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 19/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.4742 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000\n",
            "Epoch 19: saving model to trained_models/Weights-e019--v_loss_0.72742.hdf5\n",
            "1/1 [==============================] - 0s 217ms/step - loss: 0.4742 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 0.7274 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 20/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.4588 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000\n",
            "Epoch 20: saving model to trained_models/Weights-e020--v_loss_0.72951.hdf5\n",
            "1/1 [==============================] - 0s 190ms/step - loss: 0.4588 - accuracy: 1.0000 - f1_m: 1.0000 - precision_m: 1.0000 - recall_m: 1.0000 - val_loss: 0.7295 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n"
          ]
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "dv_model = tf.keras.Sequential([\n",
        "                                tf.keras.layers.Embedding(input_dim=(len(my_tokenizer.word_index.keys()) + 1), \n",
        "                                                          output_dim=150, input_length=60), # Combining the embedding layer\n",
        "                                tf.keras.layers.GRU(10, return_sequences=True, activation='relu'), # Usage of Recurrent layer\n",
        "                                tf.keras.layers.GRU(5, activation='relu'),\n",
        "                                tf.keras.layers.Dense(1,activation='sigmoid') # Lastly, we add another layer in here with activation function sigmoid\n",
        "])\n",
        "\n",
        "dv_model.compile(loss='bce', metrics=['accuracy', f1_m,precision_m, recall_m])\n",
        "\n",
        "# for doc_vecs input\n",
        "dv_history = dv_model.fit(x_train_dv, y_train_dv, epochs=20, validation_split=0.1, shuffle=True, callbacks = callbacks_list)"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "ZKbPvVwUzJ38",
        "outputId": "fbb49f63-e28f-4ec1-af25-45b085566068"
      },
      "execution_count": 16,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "Epoch 1/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6921 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 1: saving model to trained_models/Weights-e001--v_loss_0.69352.hdf5\n",
            "1/1 [==============================] - 8s 8s/step - loss: 0.6921 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6935 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 2/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6910 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 2: saving model to trained_models/Weights-e002--v_loss_0.69387.hdf5\n",
            "1/1 [==============================] - 0s 169ms/step - loss: 0.6910 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6939 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 3/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6904 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 3: saving model to trained_models/Weights-e003--v_loss_0.69424.hdf5\n",
            "1/1 [==============================] - 0s 269ms/step - loss: 0.6904 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6942 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 4/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6900 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 4: saving model to trained_models/Weights-e004--v_loss_0.69464.hdf5\n",
            "1/1 [==============================] - 0s 100ms/step - loss: 0.6900 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6946 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 5/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6897 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 5: saving model to trained_models/Weights-e005--v_loss_0.69503.hdf5\n",
            "1/1 [==============================] - 0s 109ms/step - loss: 0.6897 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6950 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 6/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6895 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 6: saving model to trained_models/Weights-e006--v_loss_0.69541.hdf5\n",
            "1/1 [==============================] - 0s 103ms/step - loss: 0.6895 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6954 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 7/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6893 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 7: saving model to trained_models/Weights-e007--v_loss_0.69578.hdf5\n",
            "1/1 [==============================] - 0s 102ms/step - loss: 0.6893 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6958 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 8/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6892 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 8: saving model to trained_models/Weights-e008--v_loss_0.69611.hdf5\n",
            "1/1 [==============================] - 0s 106ms/step - loss: 0.6892 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6961 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 9/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6891 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 9: saving model to trained_models/Weights-e009--v_loss_0.69640.hdf5\n",
            "1/1 [==============================] - 0s 112ms/step - loss: 0.6891 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6964 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 10/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6891 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 10: saving model to trained_models/Weights-e010--v_loss_0.69664.hdf5\n",
            "1/1 [==============================] - 0s 115ms/step - loss: 0.6891 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6966 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 11/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 11: saving model to trained_models/Weights-e011--v_loss_0.69684.hdf5\n",
            "1/1 [==============================] - 0s 103ms/step - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6968 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 12/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 12: saving model to trained_models/Weights-e012--v_loss_0.69699.hdf5\n",
            "1/1 [==============================] - 0s 100ms/step - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6970 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 13/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 13: saving model to trained_models/Weights-e013--v_loss_0.69710.hdf5\n",
            "1/1 [==============================] - 0s 98ms/step - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6971 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 14/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 14: saving model to trained_models/Weights-e014--v_loss_0.69717.hdf5\n",
            "1/1 [==============================] - 0s 105ms/step - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6972 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 15/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 15: saving model to trained_models/Weights-e015--v_loss_0.69722.hdf5\n",
            "1/1 [==============================] - 0s 100ms/step - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6972 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 16/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 16: saving model to trained_models/Weights-e016--v_loss_0.69725.hdf5\n",
            "1/1 [==============================] - 0s 106ms/step - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6973 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 17/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 17: saving model to trained_models/Weights-e017--v_loss_0.69727.hdf5\n",
            "1/1 [==============================] - 0s 100ms/step - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6973 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 18/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 18: saving model to trained_models/Weights-e018--v_loss_0.69728.hdf5\n",
            "1/1 [==============================] - 0s 103ms/step - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6973 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 19/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 19: saving model to trained_models/Weights-e019--v_loss_0.69729.hdf5\n",
            "1/1 [==============================] - 0s 110ms/step - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6973 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n",
            "Epoch 20/20\n",
            "1/1 [==============================] - ETA: 0s - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00\n",
            "Epoch 20: saving model to trained_models/Weights-e020--v_loss_0.69729.hdf5\n",
            "1/1 [==============================] - 0s 104ms/step - loss: 0.6890 - accuracy: 0.5455 - f1_m: 0.0000e+00 - precision_m: 0.0000e+00 - recall_m: 0.0000e+00 - val_loss: 0.6973 - val_accuracy: 0.5000 - val_f1_m: 0.0000e+00 - val_precision_m: 0.0000e+00 - val_recall_m: 0.0000e+00\n"
          ]
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "### **Why did we use RNN?**\n",
        "\n",
        "We stated our input is word sequences in Doc2Vec. For this reason, it is possible to say that our work is very similar to timeseries-based deep learning problems.\n",
        "Instead of time intervals, words are sorted.\n",
        "\n",
        "For this reason, we applied a model with **Recurrent Neural Network**, which is one of the first solutions that comes to mind in deep learning problems with long sequences.\n",
        "\n",
        "Although we ran the model with **GRU**, LSTM or regular RNN approaches could also be chosen."
      ],
      "metadata": {
        "id": "8rTNKVq-ChEk"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "fig = plt.figure(figsize=(21,10))\n",
        "\n",
        "plt.subplot(2,3,1)\n",
        "# summarize history for accuracy\n",
        "plt.plot(pts_history.history['accuracy'])\n",
        "plt.plot(pts_history.history['val_accuracy'])\n",
        "plt.title('model accuracy')\n",
        "plt.ylabel('accuracy')\n",
        "plt.xlabel('epoch')\n",
        "plt.legend(['train', 'val'], loc='upper left')\n",
        "# plt.show()\n",
        "\n",
        "\n",
        "plt.subplot(2,3,2)\n",
        "# summarize history for loss\n",
        "plt.plot(pts_history.history['loss'])\n",
        "plt.plot(pts_history.history['val_loss'])\n",
        "plt.title('model loss')\n",
        "plt.ylabel('loss')\n",
        "plt.xlabel('epoch')\n",
        "plt.legend(['train', 'val'], loc='upper right')\n",
        "\n",
        "plt.subplot(2,3,3)\n",
        "# summarize history for precision/recall\n",
        "plt.plot(pts_history.history['recall_m'], label=\"train recall\")\n",
        "plt.plot(pts_history.history['val_recall_m'], label=\"val recall\")\n",
        "plt.plot(pts_history.history['precision_m'], label=\"train precision\")\n",
        "plt.plot(pts_history.history['val_precision_m'], label=\"val precision\")\n",
        "plt.title('model precision/recall')\n",
        "plt.ylabel('precision/recall')\n",
        "plt.xlabel('epoch')\n",
        "plt.legend(loc='lower right')\n",
        "\n",
        "####################################################################################################################################\n",
        "print('\\n\\n\\n\\n')\n",
        "####################################################################################################################################\n",
        "\n",
        "plt.subplot(2,3,4)\n",
        "# summarize history for accuracy\n",
        "plt.plot(dv_history.history['accuracy'])\n",
        "plt.plot(dv_history.history['val_accuracy'])\n",
        "plt.title('model accuracy')\n",
        "plt.ylabel('accuracy')\n",
        "plt.xlabel('epoch')\n",
        "plt.legend(['train', 'val'], loc='upper left')\n",
        "# plt.show()\n",
        "\n",
        "\n",
        "plt.subplot(2,3,5)\n",
        "# summarize history for loss\n",
        "plt.plot(dv_history.history['loss'])\n",
        "plt.plot(dv_history.history['val_loss'])\n",
        "plt.title('model loss')\n",
        "plt.ylabel('loss')\n",
        "plt.xlabel('epoch')\n",
        "plt.legend(['train', 'val'], loc='upper right')\n",
        "\n",
        "plt.subplot(2,3,6)\n",
        "# summarize history for precision/recall\n",
        "plt.plot(dv_history.history['recall_m'], label=\"train recall\")\n",
        "plt.plot(dv_history.history['val_recall_m'], label=\"val recall\")\n",
        "plt.plot(dv_history.history['precision_m'], label=\"train precision\")\n",
        "plt.plot(dv_history.history['val_precision_m'], label=\"val precision\")\n",
        "plt.title('model precision/recall')\n",
        "plt.ylabel('precision/recall')\n",
        "plt.xlabel('epoch')\n",
        "plt.legend(loc='lower right')\n",
        "plt.show()"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/",
          "height": 708
        },
        "id": "ZA4vL-SzgDko",
        "outputId": "cf073b09-e156-4ad6-bf51-c2038753d0eb"
      },
      "execution_count": 17,
      "outputs": [
        {
          "output_type": "stream",
          "name": "stdout",
          "text": [
            "\n",
            "\n",
            "\n",
            "\n",
            "\n"
          ]
        },
        {
          "output_type": "display_data",
          "data": {
            "text/plain": [
              "<Figure size 1512x720 with 6 Axes>"
            ],
            "image/png": "iVBORw0KGgoAAAANSUhEUgAABM0AAAJcCAYAAADqwFMkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4yLjIsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy+WH4yJAAAgAElEQVR4nOzdeXhV5bn///edZGceCAkJM0EFAWkLyuCsDCqIqNVzHKq2autQreJY9dRT/dqe1vPrdKp1qFrUtqK1rVprtVZbUXACxAkBB2h2CIQEwrCTQObn98daCTshCYEke+8kn9d17StrPWt47o3tXmvd6xnMOYeIiIiIiIiIiIjsERftAERERERERERERGKNkmYiIiIiIiIiIiKtKGkmIiIiIiIiIiLSipJmIiIiIiIiIiIirShpJiIiIiIiIiIi0oqSZiIiIiIiIiIiIq0oaSbSDjN7zMx+2Ml9C81sdk/HJCIi0pbuumbtz3lERCTyetszipldYGb/6MR+D5rZf0cipp5gZieaWXHYetT/7aV7JEQ7ABERERERERHpe5xzTwBPdGK/K7uzXjP7NfCec+6h7jyv9D9qaSbSx5mZkuMiIiIiInJAeunzxFzgxdaFvfS7SBQpaSa9mt/s9WYz+8jMqszsN2aWb2YvmVmFmb1qZtlh+59uZp+Y2Q4zW2xm48O2TTazlf5xfwCSW9V1mpl94B/7lpl9uZMxzjOz980sZGYbzOzOVtuP9c+3w99+sV+eYmY/M7Ogme00s6V+WYumv2H/DrP95TvN7E9m9nszCwEXm9k0M3vbr6PEzH5lZolhxx9mZq+Y2TYzKzWz/zKzwWa2y8xywvY73My2mFmgM99dRET26A3XrDZivszMvvCvD8+b2VC/3MzsF2ZW5l/fPjazif62U81stR/bRjO76YD+wUREeqne8HtvXjfPB/1ngAoze93MRoVtd2Z2tZl9Dny+r7rMbISZPeM/K5Sb2a/88ovNbKm/3NG1o0W30/auP2GxXWlmn/ux3GdmFrb9y8AO51yxX/+bfr3lwJ1mlmRmPzWzIv/Z50EzSwk7/gz/e4bMbJ2ZzfHLLzGzNf6/13ozu6Iz/9bSuylpJn3B2cBJwFhgPvAS8F/AILz/jV8LYGZjgSeB6/xtLwJ/NbNE8xJIzwG/AwYCf/TPi3/sZGAhcAWQA/waeN7MkjoRXxXwdWAAMA/4tpmd6Z93lB/vvX5Mk4AP/ON+ChwBHO3H9F2gsZP/JmcAf/LrfAJoAK4HcoGjgFnAVX4MGcCrwN+BocAhwD+dc5uBxcA5Yee9CHjKOVfXyThERKSlWL9mNTOzmcCP8a4DQ4Ag8JS/+WTgeP97ZPn7lPvbfgNc4ZzLACYC/9qfekVE+oje8Ht/AfADvGeED9i7G+WZwHRgQkd1mVk88ALedaIAGMae60W4jq4dzfZx/WlyGjAV+LK/3ylh204F/ha2Ph1YD+QD/wPc7ccwCe/ZZxjwfb/uacBvgZvxnqWOBwr985T59WYClwC/MLPD2/ie0ocoaSZ9wb3OuVLn3EZgCfCuc+5951w18Cww2d/vXOBvzrlX/KTPT4EUvKTUkUAA+D/nXJ1z7k/A8rA6Lgd+7Zx71znX4Jx7HKjxj+uQc26xc+5j51yjc+4jvIviCf7mrwGvOuee9Ostd859YGZxwKXAAufcRr/Ot5xzNZ38N3nbOfecX+du59x7zrl3nHP1zrlCvItcUwynAZudcz9zzlU75yqcc+/62x4HLgTwL4bn4120RUTkwMT0NauVC4CFzrmV/vXnNuAoMysA6oAMYBxgzrk1zrkS/7g6vAesTOfcdufcyv2sV0SkL+gNv/d/c8694f/Gfw/vN35E2PYfO+e2Oed276OuaXgv3292zlX5zxRL26ivo2tHuI6uP03uds7tcM4VAa/hJcCazKNl18xNzrl7nXP1QLX/Xa73v1sF8CPgPH/fb/p1v+I/S210zq0FcM79zTm3znleB/4BHNfuv670CUqaSV9QGra8u431dH95KN5bCgCcc43ABrw3C0OBjc45F3ZsMGx5FHCj3/x3h5ntAEb4x3XIzKab2Wt+U+WdwJV4b3Pwz7GujcNy8Zpet7WtMza0imGsmb1gZpvN67L5o07EAPAXvAef0XhvynY655YdYEwiIhLj16xWWsdQidciYJhz7l/Ar4D7gDIze8jMMv1dz8Z7yx/0u/sctZ/1ioj0Bb3h9775mcH/jd/W6tjwZ4qO6hoBBP2kVLv2ce0I1+71J2yfzWHLu/D/Pc1sAF5S7q12vscgIBV4L+x7/N0vhw6ejcxsrpm943cZ3YF3rctta1/pO5Q0k/5kE96PPeD1qcf7UdwIlADDwvvCAyPDljcA/+OcGxD2SXXOPdmJehcBzwMjnHNZwINAUz0bgIPbOGYr3luQtrZV4f3QN32PePb8yDdxrdYfANYCY5xzmXhNw8NjOKitwP03YU/jtTa7CLUyExGJlGhdszqKIQ2vS85GAOfcPc65I4AJeN1cbvbLlzvnzgDy8LoVPb2f9YqI9CfR/L1vblVmZul4XUA3hW0Pf6boqK4NwEjrxCD77V07Wunw+rMPpwD/cs41tPM9tuIlLQ8L+x5ZzrmmJGabz2d+l9c/47UEzHfODcBrzWat95W+RUkz6U+eBuaZ2SzzBrK/Ea9J8VvA20A9cK2ZBczsLLxmxk0eBq70W42ZmaWZN8B/RifqzQC2Oeeq/T7yXwvb9gQw28zOMbMEM8sxs0n+G6aFwM/NbKiZxZvZUf6P9WdAsl9/ALgd2Ne4BRlACKg0s3HAt8O2vQAMMbPr/DEJMsxsetj23wIXA6ejpJmISKRE65oV7kngEjOb5F9/foTXvajQzKb65w/gvcypBhr9MXguMLMsv5tRiM6Pxyki0h9F8/f+VPMmJUvEG9vsHefchnb27aiuZXgJvrv98mQzO6b1Cdq7drRRV7vXn858J1qOZ9aC/5z1MN54ZHl+XMPMrGlMtN/4dc8yszh/2zggEe+ZawtQb2Zz8cZokz5OSTPpN5xzn+K1mLoX7w3DfGC+c67WOVcLnIWXHNqGN7bAM2HHrgAuw2tOvB34wt+3M64C7jKzCrwBJpvfuPt98E/FuzhuwxuA8yv+5puAj/HGLdgG/C8Q55zb6Z/zEby3LVVAi9k023ATXrKuAu8i8YewGCrwul7Ox2vm/DkwI2z7m3gXs5XOufDm4CIi0kOieM0Kj+FV4L/x3qyX4L15bxrzJRPverIdrwtNOfATf9tFQKF5wwFciTc2jYiItCHKv/eLgDv8cx/hx9FenO3W5bfqmo83qH4R3rPJuW2cpqNrR3hdHV1/2uW3yDsFr7tlR27x43/Hv1a9Chzq170Mf5B/YCfwOjDKf2a6Fu9Zbjves9Xz+4pJej9r2T1aRGRvZvYvYJFz7pFoxyIiIiIiIl1jZo8Bxc6526MdS3fxe/X8yjk3bZ87i3TSPvsci0j/ZmZTgcOBM6Idi4iIiIiISAfuiHYA0rcoaSYi7TKzx4EzgQV+k2QREREREZGY43etFOlW6p4pIiIiIiIiIiLSiiYCEBERERERERERaaXPdM/Mzc11BQUF0Q5DRKRPeu+997Y65wZFO45YouuOiEjP0XVnb7ruiIj0jI6uOX0maVZQUMCKFSuiHYaISJ9kZsFoxxBrdN0REek5uu7sTdcdEZGe0dE1R90zRUREREREREREWlHSTEREREREREREpBUlzURERERERERERFrpM2OataWuro7i4mKqq6ujHUqPS05OZvjw4QQCgWiHIiIiIiJdoHtYERGR2NCnk2bFxcVkZGRQUFCAmUU7nB7jnKO8vJzi4mJGjx4d7XBEREREpAt0DysiIhIb+nT3zOrqanJycvr0zQaAmZGTk9Mv3kaKiIiI9HW6hxUREYkNUUmamdlCMyszs1XtbDczu8fMvjCzj8zs8C7UdeCB9iL95XuKiIiI9Af95d6uL37PSD7riIhIz4pWS7PHgDkdbJ8LjPE/lwMPRCAmERERERGRrnoMPeuIiPQJURnTzDn3hpkVdLDLGcBvnXMOeMfMBpjZEOdcSUQC7EY7duxg0aJFXHXVVft13KmnnsqiRYsYMGBAp4+pqWvg76t63T+RiETI0YfkkpmsgZZFRGTf9vse1jlwDZw6bx6LHn2YAdnZkJTRs0HGqL74rLO6fDWbKjdFNYbCrVVs31Ub1RhEJHadMmYqXx5c0O3njdWJAIYBG8LWi/2yFhcSM7sc7+0MI0eOjFhw+2PHjh3cf//9e91w1NfXk5DQ/j//iy++uF/1VNbUs6WyliufX3lAcYpI3/fSguPIHKKkmYiI7FuLe1jXCA310FhHfc0uEsxBQx001nl/m5ZdIy/+5m5oLIdQNQzqn0mzTujUsw7ExvNObUMtF714EbWNSliJSOyqa/yvfpU06xTn3EPAQwBTpkxxUQ6nTbfeeivr1q1j0qRJBAIBkpOTyc7OZu3atXz22WeceeaZbNiwgerqahYsWMDll18OQEFBAStWrKCyspK5c+dy7LHH8tZbbzFs2DD+8pe/kJKS0qKebZU1xBm8cM2xxMf1vbEhRKTrRuemRTsEERGJNY0NYQmw2uYk2K3Xf4d1675g0sRxBBLiSU5KJDsrk7VfFPLZ0uc489Ib2FBSRnVNLQuuuITLL/06xAcoOGwqK956g8qdtcw9fvw+72GlY7HwvFNcUUxtYy3XHX4dxw47Nhoh8OeVxTyy5N/8z5kTyU5LjEoMIhLbJuYX9Mh5YzVpthEYEbY+3C87YP/vr5+welOoS0G1NmFoJnfMP6zDfe6++25WrVrFBx98wOLFi5k3bx6rVq1qnlZ74cKFDBw4kN27dzN16lTOPvtscnJyWpzj888/58knn+Thhx/mnHPO4c9//jMXXnhh8/a6hkZ2VteTmpjAxGFZ3fodRURERCR6unYP6/xukw5obF6ekJfIHSdkewky17D3YRbP3d+7nlVrP+ODpf9g8ZvLmXfO11n13luMPngMxCWy8Ik/MTAnZ8897NevICcrBywOktKhrnKf97D9WLc/6/SkYCgIwLTB0zh04KERr985x+sflzIpfzznTz4y4vWLSP8Wq0mz54HvmNlTwHRgZyz38d8f06ZNa06YAdxzzz08++yzAGzYsIHPP/98r6TZ6NGjmTRpEgBHHHEEhYWFLbZv31WLc460pPieDV5EREREYkBTIqztpJhX3tjOsQYuARKSvORWXCLEB/xPIsQFIC4Oagq99QEjIW29dw877ivNZ7nn3nu7fA/bj/WqZ52mpNnIzOh0D11dEuLT0gp+cObEqNQvIv1bVJJmZvYkcCKQa2bFwB1AAMA59yDwInAq8AWwC7ikq3Xuq0VYpKSl7eketXjxYl599VXefvttUlNTOfHEE6murt7rmKSkpObl+Ph4du/e3bzunGNbVS1pSQnUxkdrMlQRERER6ZLGRqjeAVVboL4adm2DxjruOD4LGtL2jB/mjx22N/MSX3EBiE/wEl7hibCmxJgd2P1id9/D9mXReNbpSYWhQrKTsslKik6PlmdWbiQQb8z/8pCo1C8i/Vu0Zs88fx/bHXB1hMLpURkZGVRUVLS5befOnWRnZ5OamsratWt555139vv8lTX11NY3MjgzmZKtXY1WRERERLqFc34SbKv/2QK7tra/vqt8T1fJU56GHU0nivOTYAEIpEB85p7EWFMyLC4AcfFg3TeubU/fw/Zlfe1Zp6iiiFGZo6JSd31DI3/5YCOzxuUzIFVjmYlI5MVq98w+Iycnh2OOOYaJEyeSkpJCfn5+87Y5c+bw4IMPMn78eA499FCOPHL/++hvq6olIS6OzJTA3tPtiIiIiEj3qa/1El2VZV7Sq7IMqsq8pFf4ctVWb7/G+rbPk5QFaTmQNgiyR8PwqZCW662n5oLlwaDxXmLMujcZ1lk9fQ8rvUdwZ5Ajh0bnv/GSz7eytbKWsw4fFpX6RUSUNIuARYsWtVmelJTESy+91Oa2pjEfcnNzWbVqVXP5TTfd1Lxc19BIaHc9uRmJxEXhZkpERESk16uv8RJelWVQWeolviq3+AmwLXuWK8u8lmNtCaR6Ca+0QZA1AoZO9tdzvSRYWm5YUizHG0+sI2vWQCC5+7/rfuqpe1jpPXbV7aJsdxkFmQVRqf/PK4vJTg1w4qF5UalfRERJs15sW1UtDsdANVUWERER2aOxEXZv85JglaV7EmLNf/3lis3tJ8KSs/zEVx7kjYfRJ3jr6X5Zep6fCMvzBtQX6YOKKoqA6EwCsHN3Hf9YXcr5U0eQmKCxm0UkOpQ066Wcc2yvqiU9KYGkgGbNFBERkT6sbrc3MP6uci8Z1ry8vWV51dY9CbGm8cHCBVIhPd/75I6FguP89bywv3lecmxfrcFE+oHCUCFAVFqavfRxCbX1jZx1+PCI1y0i0kRJs16qorqe2oZGBmdFv+m+iIiISKc4B7VVfuKr3E94hSfDylutb/eW6zuYdTEpE1IHQspAL+E1eCKkD24jGZavFmEi+6ko5LU0G5ExIuJ1P7NyIwcPSuPLw6Mza6eICChp1muFTwAgIiIiEhXOeUmt0CZv3K/wJNheiTB/uaGmnZMZpAzwkl+pOZA5DAZ/GVKyvaRYas6ebU1JspRsSNAwFSI9JRgKkpeaR2ogNaL1FpXvYlnhNm4+5VBMYzeLSBQpadYL1dY3UlFdx6CMJE0AICIiIj2jrhoqNkGoBCpKvMRYxeawMn+9obaNg81PdvkJrgEjYcikPcmv5r9hibCUARCnISdEYkkwFIxK18xn39+IGZw5WbNmikh0KWnWC23bVYsDBqbpzaqIiIh0QkMdVIegZidUN31CYcs7vcRYRcmehNju7XufJ5AKGUMgcyiMOBIyh0DGUMjwu0Om5XoJsOQsJcBE+oBgKMjsUbMjWqdzjmfeL+aog3IYNiAlonWLiLSmpFmMSU9Pp7Kyst3tTRMAZCQHSEzQzaiIiEi/UlcNu7ZC1RaoKvf/bvG6QYYnwWpCLZNjdVX7OLF5435lDIHsUTDySD85NmRPkixjiJcMUyt3acO+7mGl99lZs5MdNTsi3tJsZdF2guW7uGbmmIjWKyLSFiXNeplQdT11DY0M1VsXERGR3q+uGqp3eLM+Vm3x/jYnxVolxnaVe8mwtsQleAmtpk9SJuTmQ3ImJA/YU9a8T2bLfZMy1DJMRFoIhoIAjMwYGdF6/7xyIymBeOZMHBzRekVE2qKkWQ+79dZbGTFiBFdffTUAd955JwkJCbz22mts376duro6fvjDH3LGGWd06nzbqmoJxMeRmaz/dCIiIlHX2OC35trh/d29o+V6i7I29mtvUHyL97o6pg3yujsOO8JbTsvx/w7asy1tkJf0Ugsw6UbdfQ8rvU9T0mxU1qiI1Vld18ALH25izsTBpCfpeUdEoq///BK9dCts/rh7zzn4SzD37g53Offcc7nuuuuabziefvppXn75Za699loyMzPZunUrRx55JKeffvo+Z4aprW+gorqOvMxkzSIjIiLS3eprvBked28L++vP/Lh7e9hy2EyQ1TsB1/45Ld4b4L65hdcAb1bI5KyW5am5YcmwXG+/uLiIfXWJYX3gHlZ6p2AoSJzFMSJ9RMTq/NfaMkLV9Zx1uCYAEJHY0H+SZlEyefJkysrK2LRpE1u2bCE7O5vBgwdz/fXX88YbbxAXF8fGjRspLS1l8OCOmyBvq6rFgIGpmgBARESkQ40NXmuuXVu95FaV/zf801zmJ8JqOxiPKZDqz/CYvWc2yJSB3nJK9p4ukK0TZIlpagEmvVJ33sNK7xQMBRmaNpRAfCBidT6zspj8zCSOPjg3YnWKiHSk/yTN9vE2rSf953/+J3/605/YvHkz5557Lk888QRbtmzhvffeIxAIUFBQQHV1dYfnaHSObVV1/gQAevMsIiL9SN1ur6XX7h3e3+odYevbWia/doUtt9cCLDHdS4A1dW0cNM5PfvlJsBbLOd5yIDmiX1mkWS+/h5XeKxgKRrRr5tbKGhZ/uoVvHjea+Di9bBCR2NB/kmZRdO6553LZZZexdetWXn/9dZ5++mny8vIIBAK89tprBIPBfZ6jYncd9Y2NDEzTBAAiItILNTZCzU4/2bUddm1vlQBrLym2vf1xv8Dr/ticAMuFvPFeV8em9dQcP/mVu2c/JcBEOqU77mGld3LOEQwFmZw3OWJ1/vXDTdQ3Os6aPDxidYqI7IuSZhFw2GGHUVFRwbBhwxgyZAgXXHAB8+fP50tf+hJTpkxh3Lhx+zxHeVUtifFxZGgCABERiba63VBZ5rXwak54bQtb3r5nHLDdYckx19j+ORPT93RzTBkAuWPC1rO9srbWEzM09pdID+mOe1jpnbbu3squ+l2MyoxcS7NnVm5k4rBMDh2cEbE6RUT2RRmYCPn44z0DuObm5vL222+3uV9l5d7jqdTUNVBZU0++JgAQEemQmc0BfgnEA4845+5utf0XwAx/NRXIc84N8Ld9A7jd3/ZD59zjkYk6RjTUe10bK0u9hFhlqf/Z0qqszGsx1p6krD0JrZRsyB61ZzllYNhy+GcARHDMHBHpvK7cw0rv1TRzZkFmQUTq+7y0go837uT7p02ISH0iIp2lpFkvsG1XLYYxME0TAIiItMfM4oH7gJOAYmC5mT3vnFvdtI9z7vqw/a8BJvvLA4E7gCl4A2G95x+7PYJfofvVVkHVFqgq9/7u2uqvb/U/ZXuSYVVbaXMMsKRMSM+D9HzIPwwOnrlnPW3QnoHwm1qBxevWQkSkt2tKmo3MHBmR+p55fyPxccbpk4ZGpD4Rkc7SnW2Ma3SO7VV1ZKYkEIhX9xMRkQ5MA75wzq0HMLOngDOA1e3sfz5eogzgFOAV59w2/9hXgDnAkz0a8YGo3QXbC2FncatEWHhizP/U7277HAkpXsIrPQ8GjILhU7wkWFMyrGk5LQ8SUyP69UREJPqCFUECcQGGpA3p8boaGh3Pvb+RE8YOIjc9qcfrExHZH30+aeac69VdGkOdnADAuXZmCBMR6T+GARvC1ouB6W3taGajgNHAvzo4dlgbx10OXA4wcmQPvn2vqYBt/4Zt68M+/nrFpr33j0/ykmBpud5n0Lg9M0Om5fotwnL3bE9M67nYRaRb9PZ72M7SPWxsCu4MMiJjBPFx8T1e1zvryynZWc335o3v8bpERPZXn06aJScnU15eTk5OTq+96SivqiUxIY70pPb/UznnKC8vJzlZs4GJiHTSecCfnHMN+3OQc+4h4CGAKVOmdO1Jb/cO2B6WGCsPS5BVlbXcNy0PBh4EB53o/R04GgaM3JMUS0yHXnqdE5G99YV72M7QPWzsKqooitgkAH9eWUxGcgKzx+dHpD4Rkf3Rp5Nmw4cPp7i4mC1btkQ7lANS19BIaaiGrJQE1m7veIDk5ORkhg/X9Mwi0q9tBEaErQ/3y9pyHnB1q2NPbHXs4m6MbY/KLXD/dNhV3rI8Y6iXEBt7ip8YO2hPgixJM4mJ9Ce9/R52f+geNvY0NDZQFCri2GHH9nhdVTX1/H3VZs6YNJTkQM+3ahMR2V99OmkWCAQYPXp0tMM4YHf9dTW/e2czb982S/37RUT2bTkwxsxG4yXBzgO+1nonMxsHZAPhU8C9DPzIzLL99ZOB23okytQcmHAmZBfsSYxlF2jsMBFp1tvvYaV327xrM7WNtRFpafbyJ5vZVdvAWYcrcSoisalPJ816s+q6Bv68spiTDxushJmISCc45+rN7Dt4CbB4YKFz7hMzuwtY4Zx73t/1POApFzaQjnNum5n9AC/xBnBX06QA3a0R467Gb3LBISMZk68WZCIiEluaZs6MRNLsmZUbGTEwhSmjsve9s4hIFChpFqP+9lEJO3fXccH0yEzzLCLSFzjnXgRebFX2/Vbrd7Zz7EJgYY8F5yvevpsXPtrEH5Zv4EdnTeSrk/V2XUREYkekkmYlO3fz5rqtXDtzTJ8eu09Eere4aAcgbVu0rIiDctM46qCcaIciIiLdaGROKn+79ji+NDyL6//wIbc98zHVdfs1H4GIiEiPKQoVkZKQwqCUQT1az3Pvb8I5OOvwvSarFhGJGUqaxaC1m0O8F9zO+dNG6q2LiEgflJ+ZzKJvTefbJx7Mk8uKOOv+tyjcWhXtsERERCgMFTIqc1SPPoc453hmZTFTRmUzKietx+oREekqJc1i0KJ3i0hMiOPsI9RlR0Skr0qIj+OWOeNYePEUNu7Yzfx7l/LSxyXRDktERPq5YCjY410zV20M8XlZpSYAEJGYp6RZjNlVW8+zKzdy6sTBDExLjHY4IiLSw2aOy+dv1x7LQXnpfPuJlfy/v35CbX1jtMMSEZF+qK6hjk2Vm3o8afbnlcUkJsQx70tDerQeEZGuUtIsxvz1w01U1NTztek9P1uNiIjEhuHZqfzxiqO45JgCHn2zkHN+/TYbd+yOdlgiItLPFFcW0+AaejRpVtfQyPMfbuKk8flkpQZ6rB4Rke6gpFmMWfRuEWPy0plaoGmXRUT6k8SEOO6Yfxj3X3A4X5RVMu+eJby2tizaYYmISD9SFCoCenbmzNc/3cK2qlpNACAivYKSZjFk1cadfFi8k69N1wQAIiL91alfGsIL1xzLkKwULnlsOf/f39dS36DumiIi0vMKQ4UAjMrouaTZM+8Xk5OWyPFje3Z2ThGR7qCkWQx54t0ikhLiOGuyBsQUEenPCnLTePaqozl/2gjuX7yOCx55l7JQdbTDEhGRPq4oVERWUhYDkgf0yPl37qrj1dVlnD5pKIF4PYqKSOzTL1WMqKyp5/kPNjL/K0PVt19EREgOxPPjs77Mz8/5Ch8V7+TUe5by1rqt0Q5LRET6sGAo2KOtzF74eBO1DY2crVkzRaSXSIh2AH2Bc45Fy4rYsavugM/xRVklVbUNfG36yG6MTEREeruzDh/OxGFZXPXESi585F2unz2Wq2ccQlycuvGLiEj3KgwVMm3wtB47/zMrNzI2P53Dhmb2WB0iIt1JSbNu8EVZJd97dlWXzzO1IJvJI3qmKbSIiPReY/Mz+MvVx/C9Zz/mZ698xvLgdn5xzlfISU+KdmgiItLy0fIAACAASURBVNJH7K7fTemu0h6bBKBwaxXvBbdz69xxGr9ZRHqNqCTNzGwO8EsgHnjEOXd3q+2jgIXAIGAbcKFzrjjigXZSyU5vnJlFl01nyqiBB3yeQLzpAiIiIm1KS0rgF+dOYtroHO786yfM+eUSbpkzjrMmD1OrMxGRGNKJZ52RwOPAAH+fW51zL0Y80FZ6eubMZ97fiBmcOUmzZopI7xHxMc3MLB64D5gLTADON7MJrXb7KfBb59yXgbuAH0c2yv1T6g/OPHxAKokJcQf8UcJMREQ6YmZ8bfpInr3qaIYNSOGmP37IV+9/k/eC26MdmoiI0OlnnduBp51zk4HzgPsjG2Xbiip6LmnW2Oh4ZmUxxx6Sy+Cs5G4/v4hIT4lGS7NpwBfOufUAZvYUcAawOmyfCcAN/vJrwHMRjXA/lVXUAJCXqW4yIiLS8w4bmsUz3z6av3y4kbtfWsvZD7zFVycP45Y54/QwIiISXZ151nFA06BeWcCmiEbYjmAoCMDIzL3HWH7ri60Et+064HOXhqop3r6bG08ee8DnEBGJhmgkzYYBG8LWi4Hprfb5EDgLr1nzV4EMM8txzpWH72RmlwOXA4wcGb0B9EtD1WQmJ5AciI9aDCIi0r/ExRlfnTyckycM5oHF63hoyXr+vmozV514MJcdf5CuSSIi0dGZZ507gX+Y2TVAGjC7vZNF8nmncGchg1IGkRZIa1G+q7aery9cRn2j69L5s1MDnHLY4C6dQ0Qk0mJ1IoCbgF+Z2cXAG8BGoKH1Ts65h4CHAKZMmdK1X/EuKA1Vk5+pN/siIhJ5aUkJ3HTKoZw7dQQ/enENP3vlM55avoHvzRvP3ImD1fVfRCT2nA885pz7mZkdBfzOzCY65xpb7xjJ552iiqI2u2au3VxBfaPjJ//xZY4bM+iAz5+RnEBqYqw+foqItC0av1obgRFh68P9smbOuU14Lc0ws3TgbOfcjohFuJ/KKmqUNBMRkagaMTCVBy48grfWbeWuv67mqidWMn30QL4/fwKHDc2KdngiIv3FPp91gG8CcwCcc2+bWTKQC5RFJMJ2BENBZoyYsVf5mpIQAEcelKMhAESk34n4RADAcmCMmY02s0S8wS+fD9/BzHLNrCm22/Bm0oxZZaEajWcmIiIx4eiDc3nhmmP54ZkT+ay0gvn3LuW2Zz6mvLIm2qGJiPQH+3zWAYqAWQBmNh5IBrZENMpWQrUhtlVva7Ol2ZqSEBlJCQzPTolCZCIi0RXxpJlzrh74DvAysAZv5phPzOwuMzvd3+1E4FMz+wzIB/4n0nF2VmOjo6xC3TNFRCR2JMTHceGRo1h80wwuPno0f1yxgRN/uphHlqyntn6v3j8iItJNOvmscyNwmZl9CDwJXOyci9pQMwBFIW/mzLYmAVhTUsG4IRnq7i8i/VJUOpU7514EXmxV9v2w5T8Bf4p0XAdi+65a6hoceRlqaSYiIrElKzXA9+dP4GvTR3DXC2v44d/WsGhZEf992gRmHJoX7fBERPqkTjzrrAaOiXRcHWmaObMgs6BFeWOjY21JiLOPGB6FqEREoi8a3TP7lNKQ191FLc1ERCRWHZKXweOXTGXhxVNwDi55dDkXP7qMtZtD0Q5NRERiQDAUxDCGZ7RMjm3Yvouq2gbGD8mMUmQiItGlpFkXlVZUA5CvMc1ERCSGmRkzx+Xz8nXH871Tx/Ne4Xbm/N8Svv3795oHeRYRkf6pMFTI0PShJMW3fKZpuj4oaSYi/ZXm/O2iLX5Ls7wMtTQTEZHYl5gQx2XHH8R/ThnOb5b+m0ffLOSlVZuZO3Ew184aowcjEZF+qChU1OYkAKtLKogzODQ/IwpRiYhEn1qadVFpyGtpptkzRUSkNxmQmsiNJx/K0ltmcO3MQ1j6+Vbm/nIJV/7uPVZvUsszEZH+wjlHMBRkZEZbkwCEKMhNIyUxPgqRiYhEn5JmXVRaUU12aoCkBF1IRESk9xmQmsgNJx/K0ltmcu2sMbz5xVZOvWcJV/xuBZ9s2hnt8EREpIdtq95GZV0lBVkFe21bUxJSC2QR6deUNOui0lCNJgEQEZFeLys1wA0njWXpLTNZMGsMb60rZ949S7n8t0qeiYj0ZU0zZ7ZuaRaqrqN4+24mKGkmIv2YkmZdVBaqZlCGumaKiEjfkJUa4Ho/eXbd7DG8vd5Lnl322xWs2qjkmYhIX9OUNCvILGhRvrakAoDxQzSemYj0X0qadZFamomISF+UlRLgutle8uz62WN5d305p92r5JmISF8TDAVJiEtgSPqQFuWaOVNERLNndklDo2NLZQ35mgRARET6qKyUAAtmj+GSYwt47M1CHlmyntNWlzJ7fD7XzR7DxGFZ0Q5RRES6IBgKMjx9OAlxLR8N15SEGJAaYLAaCIhIP6aWZl2wraqWhkanlmYiItLnZSYHuHbWGJbeOpMbThrLsn97Lc++9bhanomI9GbBiuBeXTPBnwRgcCZmFvmgRERihJJmXVAaqgYgL0NJMxER6R/Ck2c3hiXP1G1TRKT3aXSNFIWKGJnZchKAhkbHp6UV6popIv2eumd2QVmFlzRT90wREelvMpMDXDNrDN84poDH3yzkYb/b5kkT8lkwS902RUR6g7JdZdQ01DAqc1SL8n9vraK6rlGTAIhIv6eWZl1QGqoBIE/dM0VEos7M5pjZp2b2hZnd2s4+55jZajP7xMwWhZU3mNkH/uf5yEXd+zUlz5q6bWrCABGR3qMwVAiwV9JMkwCIiHjU0qwLmrpnDkpXSzMRkWgys3jgPuAkoBhYbmbPO+dWh+0zBrgNOMY5t93M8sJOsds5NymiQfcxTd02Lz6m5YQBJ0/IZ8HsMRw2VC3PRERiTXBnEGg7aZYQZ4zJT49GWCIiMUMtzbqgNFRDTloiiQn6ZxQRibJpwBfOufXOuVrgKeCMVvtcBtznnNsO4Jwri3CM/UJT8mzJLTO5fvZY3l5fzrx7lnL5b1fwySa1PBMRiSXBiiApCSnkpea1KF9TEuLgQekkJcRHKTIRkdigbE8XbKmoVtdMEZHYMAzYELZe7JeFGwuMNbM3zewdM5sTti3ZzFb45We2V4mZXe7vt2LLli3dF30flJUSYMHsMSxtlTy74ndKnomIxIpgKMiIjBHEWcvHwjUlFRrPTEQEdc/sktJQjSYBEBHpPRKAMcCJwHDgDTP7knNuBzDKObfRzA4C/mVmHzvn1rU+gXPuIeAhgClTprjIhd57NSXPmrttLl3Py5+Ucsph+Vw7S902RUSiqShUxJjsMS3KtlfVsjlUrfHMRERQS7MuKQ1Vk5+hlmYiIjFgIzAibH24XxauGHjeOVfnnPs38BleEg3n3Eb/73pgMTC5pwPub8Jbnl03ewxvrVPLMxGRaKprrKO4oliTAIiIdEBJswNU39DI1kq1NBMRiRHLgTFmNtrMEoHzgNazYD6H18oMM8vF66653syyzSwprPwYYDXSI7JSAlw3eyxLb5nJgll7kmeXa7ZNEZGI2lS5iXpXv1fSbLWSZiIizdQ98wCVV9XS6GCQxjQTEYk651y9mX0HeBmIBxY65z4xs7uAFc655/1tJ5vZaqABuNk5V25mRwO/NrNGvJdJd4fPuik9IyslwPUnjeXSY0fz2JuF/Gbpev6xupTZ4/O5bvYYJg5Tt00RkZ4UDHkzZxZkFrQoX1NSQW56EoMy1DhARERJswNUGqoGIF8XExGRmOCcexF4sVXZ98OWHXCD/wnf5y3gS5GIUfbW1G3zkmP9Mc+WrOe0e0uZPT6PBbPG8qXhSp6JiPSEpqTZyMyRLcrXlIQ0CYCIiE/dMw9QWagGgHy1NBMREemyzOQA184aw9JbZ3LjSWNZ9u9tzP/VUr752HI+Kt4R7fBERPqcYChIRmIG2UnZzWV1DY18UVbJBHXNFBEBlDQ7YKUVfkszJc1ERES6TWZygGvCkmcrgts5/Vdvculjy/lwg5JnIiLdJRgKMipjFGbWXLZ+SxW1DY0az0xExKek2QEqDdVgBrnpidEORUREpM9pTp7dMoObTh7LyqLtnHHfm1zy6DI+UPJMRKTLikJFjMrSzJkiIh1R0uwAlYWqyU1PIiFe/4QiIiI9JSM5wHdmjmHJd2dw8ymH8v6GHZx535tc/Ogy3i/aHu3wRER6per6akqqShiVsXfSLDE+joMGpUUpMhGR2KKMzwEqDVWTp0kAREREIiIjOcDVMw5h6S0zufmUQ/lwww6+ev9bfGPhMlYqeSYisl82VGzA4RiV2TJptrokxJj8dAJqGCAiAihpdsBKQzUaz0xERCTC0pMSuHrGISy5ZSbfnXMoHxXv4Kz73+LrC5fxXlDJMxGRzigKFQG00T2zQl0zRUTCKGl2gMoqqsnPVEszERGRaEhPSuCqE72WZ7fOHceqjTs5+4G3uOg377KicFu0wxMRiWmFoUKAFt0zt1TUsLWyRkkzEZEwCdEOoDeqa2ikvKqWvAy1NBMREYmmtKQErjzhYC46chS/fyfIQ2+s5z8efJtjDslhwayxTBs9MNohikgvZGYVgGtrE+Ccc706s1RUUUROcg7pienNZXsmAciIVlgiIjFHSbMDsLWyBudQ90wREZEYkZaUwBUnHMxFR43iiXeK+PUb6zjn129z9ME5LJg1hukH5UQ7RBHpRZxzfTpzVLizcK/xzJqSZhPU0kxEpJm6Zx6A0lANgLpnioiIxJjUxAQuO/4glnx3JrfPG89npZWc+9A7nPfQ27y9rjza4YlIL2FmAzv6RDu+rgqGgm0mzYZkJTMgNTFKUYmIxB61NDsApaFqQC3NREREYlVKYjzfOu4gLpg+ikXLinjw9XWc//A7TB89kAWzx3DUQTmYWbTDFJHY9R5e98y2figccFBkw+k+lbWVlFeXt5E00yQAIiKtKWl2AMr8pFlehlqaiYiIxLKUxHi+eexoLpg+kieXFfHA4nV87eF3mVbgJc+OPljJMxHZm3NudLRj6CnBiiBAi6RZTX0D67ZUMntCXrTCEhGJSUqaHYDSUA1xBjnpSpqJiIj0BsmBeC45ZjTnTxvJH5Zv4IHF67jgkXeZWpDNdbPHKnkmIu0ys2xgDNDczcQ590b0IuqaolAR0DJp9nlpJfWNTi3NRERaUdLsAJRVVDMoI4n4ON1ci4iI9CbJgXi+cXQB504dwdMrNnD/a17ybNrogdx40lhNGCAiLZjZt4AFwHDgA+BI4G1gZjTj6orCUCEAIzJGNJftmTlTSTMRkXCaCOAAlIZqNJ6ZiIhIL5YciOfrRxWw+OYTuXP+BP69tYpzH3qHCx55h/eC26IdnojEjgXAVCDonJsBTAZ2RDekrgmGggxJG0Jywp7nmTUlFSQH4ijISYtiZCIisScqSTMzm2Nmn5rZF2Z2axvbR5rZa2b2vpl9ZGanRiPO9pSGqsnLUNJMRESkt0sOxHPxMaNZ8t0Z3D5vPJ9uruDsB97mGwuX8cGGXv1cLCLdo9o5Vw1gZknOubXAoR0dsK9nHX+fc8xstZl9YmaLeiDudhWFihiZObJF2ZqSEIcOzlRPGhGRViKeNDOzeOA+YC4wATjfzCa02u124Gnn3GTgPOD+yEbZsbKKGvIzNZ6ZiIhIX5Ec8GbbfOO7M7h17jg+Kt7Bmfe9yTcfW86qjTujHZ6IRE+xmQ0AngNeMbO/AMH2du7Ms46ZjQFuA45xzh0GXNdTwbfmnKMwVEhBZkGLsjWbQ0wYkhGpMEREeo1ojGk2DfjCObcewMyeAs4AVoft44CmDvVZwKaIRtiBmvoGtlXVqqWZiIhIH5SamMCVJxzMhUeO4vG3CnnojfWcdu9STjksn+tmj9V4PyL9jHPuq/7inWb2Gt6zyd87OKQzzzqXAfc557b7dZR1e+Dt2FGzg4raihaTAGwOVbNjV51+30RE2hCN7pnDgA1h68V+Wbg7gQvNrBh4EbimrROZ2eVmtsLMVmzZsqUnYt3LlooaALU0ExER6cPSkxK4esYhLLllBtfNHsNbX5Qz95dLuPqJlXxeWhHt8EQkQszsSDPLAHDOvQ4sxhvXrD2dedYZC4w1szfN7B0zm9NB/d36vBMMeY3kwpNmmgRARKR9sToRwPnAY8654cCpwO/MbK9YnXMPOeemOOemDBo0KCKBlTUnzdTSTEREpK/LTA5w3eyxLL1lJtfMPITFn5Zx8v+9wYKn3mfdlspohyciPe8BIPz/7JV+WVckAGOAE/Geex72u4Dupbufd9pOmnkvAsYNVvdMEZHWopE02wiMCFsf7peF+ybwNIBz7m0gGciNSHT7UBaqBiBPLc1ERET6jazUADeefChLbpnJFccfzD8+KeWkn7/ODU9/QLC8KtrhiUjPMeeca1pxzjXS8RA3nXnWKQaed87VOef+DXyGl0TrccFQkHiLZ2j60Oay1SUhRgxMISM5EIkQRER6lS4lzczsGTOb11YrsA4sB8aY2WgzS8Qb6P/5VvsUAbP8OsbjJc0i0/9yH0pDamkmIiLSXw1MS+TWueNYcssMLj1mNH/7qIRZP3ud25/7uPnFmoj0KevN7FozC/ifBcD6DvbvzLPOc3itzDCzXLzumh2ds9sUhgoZnjGcQNyeBNmakhDjB6trpohIW7ra0ux+4GvA52Z2t5l1OP0ygHOuHvgO8DKwBm+WzE/M7C4zO93f7UbgMjP7EHgSuDj8DU80lYaqSYgzBqYmRjsUERERiZLc9CRuP20CS747g/OnjeSpZRs4/iev8b9/X8vOXXXRDk9Eus+VwNF4rcWKgenA5e3t3MlnnZeBcjNbDbwG3OycK+/B79CsKFTUomvm7toGCrdWaTwzEZF2dGn2TOfcq8CrZpaF1x//VTPbADwM/N451+Zdo3PuRbwB/sPLvh+2vBo4piux9ZTSUA15GUnExVm0QxEREZEoy8tM5gdnTuRbx43mF698xoOvr+OJd4J8+8RDuPjoAlIS46Mdooh0gT+z5Xn7ecy+nnUccIP/iRjnHEUVRUwdPLW57NPSChqdJgEQEWlPl8c0M7Mc4GLgW8D7wC+Bw4FXunruWFRWUc0gdc0UERGRMKNy0vi/8ybzt2uOY0rBQP7372s54Sev8ft3gtQ1NEY7PBE5QGY21sz+aWar/PUvm9nt0Y7rQJTtKmN3/W4KMguay/bMnKlJAERE2tLVMc2eBZYAqcB859zpzrk/OOeuAdK7I8BYUxqqJj9DkwCIiIjI3iYMzWThxVP545VHMXJgKrc/t4rZP3+dv3ywkcbGmBhpQkT2z8PAbUAdgHPuI/az5VmsaJo5c2TmyOayNSUh0hLjGZGdGq2wRERiWldbmt3jnJvgnPuxc64kfINzbkoXzx2TyipqNAmAiIiIdGhqwUD+eOVRLLx4CimBeBY89QHz7l3Ka5+WESPDtIpI56Q655a1KquPSiRdVBgqBNirpdm4IZkaekZEpB1dTZpNMLMBTStmlm1mV3XxnDGruq6BHbvqyM9USzMRERHpmJkxc1w+L157HL88bxJVNfVc8uhyzv31O6wo3Bbt8ESkc7aa2cGAAzCz/wBKOj4kNhWFikiKTyI/LR/wxjhbW1KhrpkiIh3oatLsMufcjqYV59x24LIunjNmbamoAbxBf0VEREQ6Iy7OOGPSMF694QR+cMZh/Lu8iv948G2+9fhy1m4ORTs8EenY1cCvgXFmthG4Dm9GzV4nGAoyImMEceY9AhZv301FTb0mARAR6UCXZs8E4s3M/BlgMLN4ILHrYcWm0lA1gLpnioiIyH5LTIjjoqMKOPuI4Tz6ZiEPvr6Oub9cwpmThnHDSWMZMVBjConEEv/Z5irn3GwzSwPinHMV0Y7rQAUrghycdXDz+urmSQCUNBMRaU9XW5r9HfiDmc0ys1nAk35Zn1Qa8luaaSIAEREROUCpiQlcPeMQlnx3BlccfzAvflzCrJ+/zt0vrSVUXRft8ETE55xrAI71l6t6c8KsvrGeDRUb9poEwAzGDVb3TBGR9nS1pdktwBXAt/31V4BHunjOmKWWZiIiItJdBqQmcuvccXzj6FH85OVP+fUb63h6xQaumz2G86eNJBDf1XebItIN3jez54E/AlVNhc65Z6IX0v4rqSqhvrF+r0kACnLSSE3s6iOhiEjf1aVfSOdcI/CA/+nzyipqCMQb2amBaIciIiIifcSQrBR+fs4kLj1mNP/ztzV8/y+f8Nhbhdw2dzyzx+dhplntRKIoGSgHZoaVOaBXJc2CoSBAq5ZmFUwcpq6ZIiId6dIrTDMbY2Z/MrPVZra+6dNdwcWaslA1eRnJunkVEYlRZjbHzD41sy/M7NZ29jnHv259YmaLwsq/YWaf+59vRC5qEc/EYVksumw6j3x9CgCX/XYF5z/8Dqs27oxyZCL9j5mdb2Y5zrlL2vhcGu349ldT0mxU5igAKqrrKNq2i/GDlTQTEelIV9v9P4rXyqwemAH8Fvh9V4OKVaUV1eRnajwzEZGeZmYLzCzTPL8xs5VmdvI+jokH7gPmAhOA881sQqt9xgC3Acc45w7DmwUNMxsI3AFMB6YBd5hZdrd/MZF9MDNmT8jn5euO5wdnHMZnpZWcdu9SbvjDB2zasTva4Yn0JyOBP5rZEjO708ymWy9+cx4MBUkPpJOTnAPAp5u94dk0CYCISMe6mjRLcc79EzDnXNA5dycwr+thxabSUI3GMxMRiYxLnXMh4GQgG7gIuHsfx0wDvnDOrXfO1QJPAWe02ucy4D7n3HYA51yZX34K8Ipzbpu/7RVgTvd8FZH9F4j3ZtpcfPOJXHnCwbzwcQkzfrqYn778KZU19dEOT6TPc879r3NuJnAq8CFwKbDSzBaZ2dfNLD+6Ee6fYCjIyMyRzT1m1jTNnDlUSTMRkY50NWlWY2ZxwOdm9h0z+yqQ3g1xxaTSULVmzhQRiYymt/mnAr9zzn0SVtaeYcCGsPVivyzcWGCsmb1pZu+Y2Zz9OBYzu9zMVpjZii1btnTyq4gcuMzkALfOHcc/bziBUw4bzK9e+4ITf7KYRe8WUd/QGO3wRPo851yFc+5Z59wVzrnJwA+BQXg9bHqNYCjY3DUTYHVJBZnJCQzNUoMAEZGOdDVptgBIBa4FjgAuBPrkODC7axuoqK4nTy3NREQi4T0z+wde0uxlM8sAuiNDkACMAU4EzgceNrMBnT3YOfeQc26Kc27KoEGDuiEckc4ZMTCVe86fzLNXHU1BTir/9ezHnHrPEl77tAznXLTDE+nTzGyYmR1tZscDucBy59wp0Y6rs2obatlUualF0mxNSYjxQzI1VrOIyD4c8OyZ/tgx5zrnbgIqgUu6LaoYVFZRDaDumSIikfFNYBKw3jm3yx9zbF/XmY3AiLD14X5ZuGLgXedcHfBvM/sML4m2ES+RFn7s4gOOXqSHTB6ZzR+vPIqXP9nMj19ayyWPLufYQ3L53rzxGptIpAeY2f8C5wKrgQa/2AFvRC2o/bShYgMO15w0a2h0fLq5gnOnjtjHkSIicsAtzZxzDcCx3RhLTCsN1QBoIgARkcg4CvjUObfDzC4Ebgf2NYXgcmCMmY02s0TgPOD5Vvs8h58cM7NcvO6a64GXgZPNLNufAOBkv0wk5pgZcyYO4ZXrT+C/T5vAxxt3Mu+eJdz2zMdsrayJdngifc2ZwKHOuVOdc/P9z+nRDmp/NM+cmeElzYLlVeyua2CCEu0iIvvU1e6Z75vZ82Z2kZmd1fTplshiTGlILc1ERCLoAWCXmX0FuBFYxz7Gj3HO1QPfwUt2rQGeds59YmZ3mVnTA87LQLmZrQZeA252zpU757YBP8BLvC0H7vLLRGJWYkIc3zx2NK/ffCLfOLqAP67YwIyfLObXr6+jpr5h3ycQkc5YDwSiHURXNCXNRmaOBGBNiWbOFBHprAPunulLBsqBmWFlDnimi+eNOc1JswwlzUREIqDeOefM7AzgV86535jZN/d1kHPuReDFVmXfD1t2wA3+p/WxC4GFXY5cJMIGpCZyx/zDuGD6KH704hp+/NJaFi0r4r9OHc/JE/I1ZpFI1+wCPjCzfwLNTTmdc9dGL6T985VBX+HKr1xJVlIW4I1nFh9njMnvs/O3iYh0my4lzZxzfXocs3BlFTUkJcSRmdLVPKOIiHRChZndBlwEHOfP1Nyr3/SL9LRD8tJZePFUXv9sCz94YTVX/O49jjooh+/Pn6AWJSIH7nn27urfqxyefziH5x/evL6mJMRBuWkkB+KjGJWISO/QpQyQmT2K17KsBefcpV05bywqDVWTl5mkt7UiIpFxLvA14FLn3GYzGwn8JMoxifQKJ4wdxDELjmPRsiJ+/spnzLtnCedOHcmNJ48lN11js4rsD+fc4/44mWP9ok/9yWR6rTUlIaYUDIx2GCIivUJXm029ELacDHwV2NTFc8ak0lC1umaKiESInyh7AphqZqcBy5xzHY5pJiJ7JMTH8fWjCjjjK8P4v39+xu/eDvLCh5u4ZtYhfOPoApIS1MJEpDPM7ETgcaAQMGCEmX3DOddrZs8Mt2NXLZt2Vqv1qYhIJ3VpIgDn3J/DPk8A5wBTuie02FJWUaNJAEREIsTMzgGWAf+Jd21518z+I7pRifQ+WakB7pj//7N33+FVlPn7x9+fhEAEQgtgoSUovSUQmoiyoogtYgEsqKyrgg1ZV13UVfm5fnd11wpiRRRdVIqroqIgKAoiaEAQpEgxCIgJRUgiJpQ8vz8ysCEkISQnZ85J7td1nYszM8/M3GdSHuaTZ2ba8fHI0+kaX49/zFhNvye/YNb3v5B3iz8ROYrHgX7OuTOcc6cD5wBP+pyp1P73EIAYn5OIiISHsj49s6AWQMMAbzMkpGfk0LCWLmkQEQmS+4CuzrlrnXPXAN2A+33OJBK2Dt7vbOJ13YiKjODG1xdz1fhFrNqa4Xc0kVAX5Zxbc3DCOfcDYXyPzYM/82010kxEpETKVDQzdeUojgAAIABJREFUs0wzyzj4At4H/hqYaKEjK2c/WTn7NdJMRCR4Ipxz6fmmdxD4P/SIVDpntGzAx7f35qGL2rFyawbnj5nHPf9dzvasnKOvLFI5pZjZeDPr471eAlL8DlVaq7ZmEFujKg1iNBhARKQkyvr0zEoxrjc9IxuA4zXSTEQkWD42s5nAm970YGCGj3lEKoyi7nd2+1ktuPbUOKIiVZ8Wyecm4BZghDc9D3jWvzhls+qXDNqcWEsPNxMRKaGyjjS72Mxq55uuY2YDyh4rtKRl5P31taEeBCAiEhTOubuAF4GO3utF51yFG8ks4qf89zvrEleXhz9cxblPz2Pe2m1+RxMJGc65HOfcE865S7zXk865sByauf9ALj+kZel+ZiIix6Csf0p80Dm3++CEc24X8GAZtxly0jM10kxEJNi8h8zc4b3e8TuPSEV1SsOavDK0Ky9fm8S+A7lc/fLXDHs9hU079/gdTcQ3ZjbF+3e5mX1X8OV3vtLYsP039u7P1ZMzRUSOQZkuz6TwoltZtxly0g+ONNM9zUREypWZZQKFPdLPAOec0//0RcqBmdG3zfH0OqU+L8//kWc+XcdZaz5n+Bknc1Ofk4mOivQ7okiw3e79e4GvKQLo4EMAVDQTESm5so40SzGzJ8zsZO/1BLA4EMFCSVpGNsdFRRJTrcLVA0VEQopzLsY5V6uQV4wKZiLlLzoqklv+cApz/nIGZ7c9nqfnrKXv45/z0fKtOFdYPVukYnLObfXebgc2Oec2AtWATsDPvgUrg5VbM4iKNE5uUNPvKCIiYaOsRbPbgL3AZOAtIJu8G2VWKGmZORxfq5pumCkiIiKVwkl1juOZKzvz1o09iImuwk2TljDk5UWsTcv0O5pIsH0BRJtZI2AWcDXwqq+JSmn11kxOaRhD1Sp62IeISEmV6Temc+4359wo51ySc66rc+5e59xvgQoXKtIysnVppoiIiFQ6PZrH8sFtp/H/ktuxfPNuzn16Hn//YCUZ2fv8jiYSLOac2wNcAjzrnBsItPM5U6ms2pqhhwCIiByjsj498xMzq5Nvuq6ZzSx7rNCSnpHN8SqaiYiISCVUJTKCa0+N47M7+zAwqQkTvvyRMx+by5SUTeTm6pJNqfDMzHoCVwEfevPC7iZ/O7JySM/Moa3uZyYickzKOja3vvfETACcc78CDcu4zZDinCMtI4eGMXpypoiIiFResTWr8c9LOjD9ltNoWq86d0/7jkueW8CyTbuOvrJI+BoJ3AO845z73syaA5/5nOmYrdqad2m1HgIgInJsylo0yzWzpgcnzCyOwp96Fraycvbz+74DHF9LRTMRERGRDo1rM234qTw+sBObf/2dAc9+yV+nfcf2rBy/o4kEnHPuc+dcsnPuUW96g3NuhN+5jpWenCkiUjplfRzkfcB8M/scMKA3cGOZU4WQtIy8/wDq8kwRERGRPBERxqVdGtOv3fGM/XQdE+b/yEcrtnJ3/9Zc2a0pERF6eJKENzN7yjk30szep5BBAc65ZB9ildqqrRkcX6sa9WpU9TuKiEhYKVPRzDn3sZklkVco+xZ4F/g9EMFCRXpGNgANY1Q0ExEREckvJjqKe89rw6CkJtz/7gr+9u4Kpi7ezP8NaE/7RrX9jidSFq97/z7ma4oAWbk1Q6PMRERKoawPArgemAP8BbiTvM5ldAnW629ma8xsnZmNKmT5k2a21Hv9YGa+3SwjLTOvaKbLM0VEREQKd0rDmrxxQ3eeGpzAll/3kPzMfEZP/55MPWVTwpRzbrH3NgWY512m+TkwH/imuHWPdq6Tr92lZua8QQjlZu/+XNZvy1LRTESkFMp6T7Pbga7ARufcH4BEoNgCl5lFAuOAc4G2wBVm1jZ/G+fcn51zCc65BGAs8N8y5iy1g5dnNtTlmSIiIiJFMjMGJDZizh19uKp7MyZ+lUrfxz/n/WU/41yFuuWtVC5zgOr5po8DZhfVuCTnOl67GPLOpRYFNG0h1qVnse+AU9FMRKQUylo0y3bOZQOYWTXn3Gqg1VHW6Qas826iuRd4C7iomPZXAG+WMWeppWVkU6NqJDWrlfX2byIiIiIVX+3qUfx9QHvevbkXDWtV47Y3v+WaCV/z4/bf/I4mUhrRzrmsgxPe++rFtC/puc7fgUeB7ECGLczBhwC0PTGmvHclIlLhlLVottnM6pB3L7NPzOw9YONR1mkEbMq/DW/eEcysGRAPfFrE8hvNLMXMUrZt23bM4UsiPSNHDwEQEREROUadmtThvVtOY/SFbVn60y7OeeoLnpr9A9n7DvgdTeRY/GZmnQ9OmFkXir+H81HPdbztNXHOfXi0nQfifCc9M4caVSOJi61RqvVFRCqzsj4I4GLv7Wgz+wyoDXxc5lT/czkwzTlX6P+unHMvAi8CJCUllcu4//TMbBrqfmYiIiIixywywhjaK57zOpzI3z9cxVOz1/Lut1v4+4D29G7RwO94IiUxEphqZj8DBpwADC7txswsAngCGFqS9oE437mpz8lc3zueKpFlHS8hIlL5BOw3p3dzzOneMOTibAGa5Jtu7M0rzOX4eGkm5N3TTCPNREREREqvYa1oxl6RyOt/6oaZcfXLX3PLG0tIyyj3K9NEysQ59w3QGrgJGA60yfeQgMIc7VwnBmgPzDWzVKAHML28HwYQpYKZiEip+PHb8xughZnFm1lV8gpj0ws2MrPWQF3gqyDnO8Q5R1pGtopmIiIiIgHQu0UDPrq9N38+qyWfrEyj7+Of88qXP7L/QK7f0UQKZWbVgb8CtzvnVgBxZnZBMasUe67jnNvtnKvvnItzzsUBC4Fk51xK+X0KEREpraAXzZxz+4FbgZnAKmCKc+57M3vIzJLzNb0ceMv5+LiljN/3k7M/l4YxujxTREREJBCioyK5/awWzBp5OolN6/D/3l/JReO+ZOmmYh/ALuKXV4C9QE9vegvwcFGNj+FcR0REwoAvj4R0zs0AZhSY90CB6dHBzFSYtMy8SwY00kxEREQksOLq1+C167rx4fKtPPT+Si5+9kuGnhrHXee0onpVPbVcQsbJzrnBZnYFgHNuj5lZcSuU5Fwn3/w+gQoqIiKBp4vbi3HwPhsaaSYiIiISeGbGBR1PYs5fzuDqHs145ctUzn16Hos27PA7mshBe83sOMABmNnJQI6/kUREJFhUNCtGekZef6iRZiIiIiLlJyY6iocuas+bN/TAORj84kIefG8Fv+Xs9zuayIPAx0ATM5sEzAHu9jeSiIgEi4pmxTh4eWbDWhppJiIiIlLeep4cy8cjezP01DgmfrWR/k9/wYL12/2OJZWUmUWQ92CyS4ChwJtAknNuro+xREQkiFQ0K0Z6Rg4x0VV0Xw0RERGRIKletQqjk9sx+cYeRJhx5UuLuP9djTqT4HPO5QJ3O+d2OOc+dM594JxTFVdEpBJR0awYaRnZujRTRERExAfdm8fy8e2nc12veP6zaCPnPPUFC9apXiFBN9vM7jSzJmZW7+DL71AiIhIcKpoVI69opkszRUTCgZn1N7M1ZrbOzEYVsnyomW0zs6Xe6/p8yw7kmz89uMlFpCjHVY3kgQvbMnVYT6IiI7hy/CLue2c5WRp1JsEzGLgZ+BxIyfcSEZFKQEWzYqRl5NAwRiPNRERCnZlFAuOAc4G2wBVm1raQppOdcwnea3y++b/nm58cjMwiUnJJcfWYMaI3N/SO542vf+KcJ79g/lqNOpOgaEte/7IMWAqMBdr5mkhERIJGRbMiOOfYlpmjhwCIiISHbsA659wG59xe4C3gIp8ziUgAHVc1kvvOb8u04T2pViWCIS8v4p7/fkdm9j6/o0nFNhFoA4whr2DW1psnIiKVgIpmRdi1Zx97D+RyvEaaiYiEg0bApnzTm715BV1qZt+Z2TQza5JvfrSZpZjZQjMbUNROzOxGr13Ktm3bAhRdRI5Fl2b1mHF7b4ad3pzJ32zinCe/4PMf9PMo5aa9c+5659xn3usGoL3foUREJDhUNCtCWmY2gB4EICJScbwPxDnnOgKfcPhIgWbOuSTgSuApMzu5sA045150ziU555IaNGhQ/olFpFDRUZHcc14bpt10KsdVjeTaCV/z12nfkaFRZxJ4S8ysx8EJM+uO7mkmIlJpqGhWhLSMHAA9CEBEJDxsAfKPHGvszTvEObfDOZfjTY4HuuRbtsX7dwMwF0gsz7AiEhidm9blwxG9GX7GyUxdnDfqTE/YlADrAiwws1QzSwW+Arqa2XIz+87faCIiUt5UNCtCWoZGmomIhJFvgBZmFm9mVYHLgcOegmlmJ+abTAZWefPrmlk17319oBewMiipRaTMoqMiGXVua/57cy+Oi4rkyvGLePiDlWTvO+B3NKkY+gPxwBneK96bdwFwoY+5REQkCKr4HSBUpXtFswYxGmkmIhLqnHP7zexWYCYQCUxwzn1vZg8BKc656cAIM0sG9gM7gaHe6m2AF8wsl7w/Jj3inFPRTCTMJDSpwwcjTuMfM1Yxfv6PzFu7nacuT6DNibX8jiZhzDm30e8MIiLiHxXNipCWkUPt46KIjor0O4qIiJSAc24GMKPAvAfyvb8HuKeQ9RYAHco9oIiUu+pVq/DwgA70bXM8d0/7joue+ZK/9GvJ9b2bExlhfscTERGRMKPLM4uQnpmt+5mJiIiIhKE/tGrIzJGn84fWDfjnR6u58qWFbP51j9+xREREJMyoaFaEtIwc3c9MREREJEzVq1GV54d04V+XdWTFlt2c+9Q8/rtkM845v6OJiIhImFDRrAjpGdk0jFHRTERERCRcmRmDkprw8cjTaXVCDHdMWcatb3zLrj17/Y4mIiIiYUBFs0Lk5jrSM3N0eaaIiIhIBdCkXnUmD+vJ3f1bMWvlL5zz1BfMW7vN71giIiIS4lQ0K8TOPXvZn+t0eaaIiIhIBREZYdzc5xTeubkXMdFRXP3y14ye/j3Z+w74HU1ERERClIpmhUjLyAagYYxGmomIiIhUJO0b1eaD205j6KlxvLoglQvGzmfFlt1+xxIREZEQpKJZIdIzcwBoqJFmIiIiIhVOdFQko5Pb8fqfupGZvY8B475k3GfrOJCrhwSIiIjI/6hoVoh0b6SZ7mkmIiIiUnH1btGAmSNP55z2J/DvmWsY/MJXbNq5x+9YIiIiEiJUNCtEWkbeSLMGujxTREREpEKrU70qz1yRyFODE1jzSybnPT2Pd7/d4ncsERERCQEqmhUiLSObejWqUq1KpN9RRERERKScmRkDEhsx4/betD4xhpGTl3L7W9+Skb3P72giIiLiIxXNCpGWkaOHAIiIiIhUMk3qVefNG3pwx9kt+eC7rZz39DxSUnf6HUtERER8oqJZIdIzs/UQABEREZFKqEpkBCP6tmDKsJ6YwaAXvuLJT35g/4Fcv6OJiIhIkKloVoi0jGyO10gzERERkUqrS7O6zBjRmwGJjXh6zloGvfAVP+3QQwJEREQqkyp+Bwg1B3Id27P2crxGmomIiIhUajHRUTwxKIE+rRpy3zvLOW/MPB66qB0XJzbCzPyOJyIiIWLfvn1s3ryZ7Oxsv6NIMaKjo2ncuDFRUVElXkdFswJ2/JbDgVzH8bU00kxEREREILnTSXRuWoc7Ji/jjinLmLtmGw9f3J5a0SX/T7eIiFRcmzdvJiYmhri4OP1RJUQ559ixYwebN28mPj6+xOvp8swC0jNyAHRPMxERERE5pHHd6rx5Yw/u7NeSD5dv5dyn5vGNHhIgIiJAdnY2sbGxKpiFMDMjNjb2mEcDqmhWQFpG3gHU5ZkiIiIikl9khHHrmS2YNrwnkRHG4Be+4olZa/SQABERUcEsDJTma6SiWQFp3kgzXZ4pIiIiIoVJbFqXGbf35uLExoz5dB0DX/iKjTt+8zuWiIiIBJiKZgWkZWRjBvVrqmgmIiIiIoWrWa0Kjw/qxNgrElmXnsV5T8/j7cWbcc75HU1ERCqZXbt28eyzz5Zq3fPOO49du3YFONGxi4uLY/v27QDUrFnT5zT/o6JZAemZOcTWqEpUpA6NiIiIiBTvwk4n8fHI02nXqDZ/mbqM299aSmb2Pr9jiYhIJVJc0Wz//v3Frjtjxgzq1KlTqv0ebdsVgSpDBaRnZNMwRvczExEREZGSaVTnON684X8PCbhw7HxWbNntdywREakkRo0axfr160lISOCuu+5i7ty59O7dm+TkZNq2bQvAgAED6NKlC+3atePFF188tO7BEV6pqam0adOGG264gXbt2tGvXz9+//33I/Y1dOhQhg8fTvfu3bn77rtZv349/fv3p0uXLvTu3ZvVq1cDkJaWxsUXX0ynTp3o1KkTCxYsKDZHqKrid4BQk5aZrfuZiYiIiMgxOfiQgO7NY7ntjW+55NkF3Hd+G67p2Uw3hxYRqUT+3/vfs/LnjIBus+1JtXjwwnZFLn/kkUdYsWIFS5cuBWDu3LksWbKEFStWEB8fD8CECROoV68ev//+O127duXSSy8lNjb2sO2sXbuWN998k5deeolBgwbx9ttvM2TIkCP2t3nzZhYsWEBkZCR9+/bl+eefp0WLFixatIibb76ZTz/9lBEjRnDGGWfwzjvvcODAAbKyskqcI5T4MtLMzPqb2RozW2dmo4poM8jMVprZ92b2RrCypWXk6MmZIiIiIlIqXePqMeP23pzWoj4PTv+em/6zhN2/63LNyuRo5zpmdod3nvOdmc0xs2Z+5BSRiq1bt26HCmYAY8aMoVOnTvTo0YNNmzaxdu3aI9aJj48nISEBgC5dupCamlrotgcOHEhkZCRZWVksWLCAgQMHkpCQwLBhw9i6dSsAn376KTfddBMAkZGR1K5du8Q5QknQR5qZWSQwDjgb2Ax8Y2bTnXMr87VpAdwD9HLO/WpmDYORbf+BXLZn5dBQRTMRERERKaV6Naoy/pokXp7/I49+vJrzx8zjmSs7k9CkdPeMkfBRknMd4FsgyTm3x8xuAv4FDA5+WhEpD8WNCAumGjVqHHo/d+5cZs+ezVdffUX16tXp06cP2dnZR6xTrdr/rrqLjIws9PLM/NvOzc2lTp06h0a4HU1Jc4QSP0aadQPWOec2OOf2Am8BFxVocwMwzjn3K4BzLj0YwbZn7cU5aBijyzNFREREpPQiIowbTm/OlOE9cQ4ue24B4+dt0NM1K76jnus45z5zzu3xJhcCjYOcUUQqmJiYGDIzM4tcvnv3burWrUv16tVZvXo1CxcuDMh+a9WqRXx8PFOnTgXAOceyZcsA6Nu3L8899xwABw4cYPfu3eWWozz5UTRrBGzKN73Zm5dfS6ClmX1pZgvNrH9hGzKzG80sxcxStm3bVuZg6Zl5FU5dnikiIiIigdC5aV1mjOjNma0b8vCHq7h+Ygq//rbX71hSfkpyrpPfn4CPiloY6PMdEamYYmNj6dWrF+3bt+euu+46Ynn//v3Zv38/bdq0YdSoUfTo0SNg+540aRIvv/wynTp1ol27drz33nsAPP3003z22Wd06NCBLl26sHLlynLNUV4s2H/tMrPLgP7Oueu96auB7s65W/O1+QDYBwwi7y8vXwAdnHO7itpuUlKSS0lJKVO2T1amccNrKUy/tRcdG2v4vIjIQWa22DmX5HeOUBKIfkdEKg/nHBMXpPKPGauJrVmVsVckkhRXz+9YIStc+52SnOvkazsEuBU4wzmXc7Rtq98RCV2rVq2iTZs2fseQEijsa1Vcn+PHSLMtQJN80429efltBqY75/Y5534EfgBalHewtAyNNBMRERGRwDMzhvaK5+2bTiUqMoLBLy5k3GfryM3V5ZoVTEnOdTCzs4D7gOSSFMxERMQffhTNvgFamFm8mVUFLgemF2jzLtAHwMzqk3e55obyDpaekU2EQWyNquW9KxERERGphDo0rs0HI06jf/sT+PfMNVz7ytdsz1LNpAI56rmOmSUCL5BXMAvKvZtFRKR0gl40c87tJ28Y8kxgFTDFOfe9mT1kZsles5nADjNbCXwG3OWc21He2dIycqhfsxpVIv2oJYqIiIhIZVArOopnrkjk/y5uz6Ifd3Le0/P4an25/1dXgqCE5zr/BmoCU81sqZkVHEAgIiIhooofO3XOzQBmFJj3QL73DrjDewVNWma2Ls0UERERkXJnZlzVvRmJTepy6xtLuGr8Qkb0bcFtZ7YgMsL8jidlUIJznbOCHkpEREpFQ6ryScvIoWFMNb9jiIiIiEgl0fakWrx/22lclNCIp2avZcj4RaR799kVERERf6lols+2zGwaaqSZiIiIiARRjWpVeGJQJ/51WUe+3fQr542Zz4J12/2OJSIiUumpaObZdyCX7Vl7Ob6WRpqJiIQrM+tvZmvMbJ2ZjSpk+VAz2+bdQ2apmV2fb9m1ZrbWe10b3OQiUtmZGYOSmvDeLadR+7gqDHl5EWPmrOWAnq4pIiLloGbNmkHb16uvvsqtt94KwOjRo3nssceCtu+yUtHMsy0z76lFuqeZiEh4MrNIYBxwLtAWuMLM2hbSdLJzLsF7jffWrQc8CHQHugEPmlndIEUXETmk1QkxTL/1NC7sdBJPfPIDQ/V0TRER8YFzjtzcXL9j+E5FM0+ad+8IjTQTEQlb3YB1zrkNzrm9wFvARSVc9xzgE+fcTufcr8AnQP9yyikiUqwa1arw1OAE/nFxBxb9uJPzx8zj6x93+h1LRERC1KhRoxg3btyh6YOjubKysujbty+dO3emQ4cOvPfee8VuJzU1lVatWnHNNdfQvn17Nm3axL///W+6du1Kx44defDBBw+1fe211+jYsSOdOnXi6quvBuD999+ne/fuJCYmctZZZ5GWllY+HziIfHl6ZihKy8j7C17DGI00ExEJU42ATfmmN5M3cqygS83sdOAH4M/OuU1FrNuo4IpmdiNwI0DTpk0DFFtE5EhmxpXdm9KpSW1umbSEK15ayJ39WjHs9OZE6OmaIiKh66NR8MvywG7zhA5w7iNFLh48eDAjR47klltuAWDKlCnMnDmT6Oho3nnnHWrVqsX27dvp0aMHycnJmBXdj6xdu5aJEyfSo0cPZs2axdq1a/n6669xzpGcnMwXX3xBbGwsDz/8MAsWLKB+/frs3Jn3h53TTjuNhQsXYmaMHz+ef/3rXzz++OOBPRZBpqKZJz0zb6RZQ400ExGpyN4H3nTO5ZjZMGAicGZJV3bOvQi8CJCUlKQbDYlIuWt3Um3ev+00Rr29nEc/Xs3XP+7giUEJ1K1R1e9oIiISIhITE0lPT+fnn39m27Zt1K1blyZNmrBv3z7uvfdevvjiCyIiItiyZQtpaWmccMIJRW6rWbNm9OjRA4BZs2Yxa9YsEhMTAcjKymLt2rUsW7aMgQMHUr9+fQDq1asHwObNmxk8eDBbt25l7969xMfHl/MnL38qmnnSM3KIjDBia6hoJiISprYATfJNN/bmHeKc25Fvcjzwr3zr9imw7tyAJxQRKYWY6CieuTKR7gvr8fAHqzh/zDyeuaoznZvq1osiIiGnmBFh5WngwIFMmzaNX375hcGDBwMwadIktm3bxuLFi4mKiiIuLo7s7Oxit1OjRo1D751z3HPPPQwbNuywNmPHji103dtuu4077riD5ORk5s6dy+jRo8v2oUKA7mnmScvIpkHNakRquLuISLj6BmhhZvFmVhW4HJiev4GZnZhvMhlY5b2fCfQzs7reAwD6efNEREKCmXFNzzim3dSTyEhj0PNfMX7eBpzToFcREcm7RPOtt95i2rRpDBw4EIDdu3fTsGFDoqKi+Oyzz9i4ceMxbfOcc85hwoQJZGVlAbBlyxbS09M588wzmTp1Kjt25P09+uDlmbt376ZRo7w7nEycODFQH81XGmnmScvM0UMARETCmHNuv5ndSl6xKxKY4Jz73sweAlKcc9OBEWaWDOwHdgJDvXV3mtnfySu8ATzknNNdt0Uk5HRsXIcPbuvNXVOX8fCHq/j6x538+7JO1K4e5Xc0ERHxUbt27cjMzKRRo0aceGLe34mvuuoqLrzwQjp06EBSUhKtW7c+pm3269ePVatW0bNnTwBq1qzJf/7zH9q1a8d9993HGWecQWRkJImJibz66quMHj2agQMHUrduXc4880x+/PHHgH/OYLOK8teppKQkl5KSUur1+z/1BU3qVeela5ICmEpEpGIws8XOOf2CzKes/Y6ISFk453h5/o888tFqTqgdzbNXdaZj4zp+xwoY9TtHUr8jErpWrVpFmzZt/I4hJVDY16q4PkeXZ3rSMrI10kxEREREwoKZcX3v5kwZ3pPcXMdlz33Fa1+l6nJNERGRAFLRDMjZf4Bf9+zj+Jhov6OIiIiIiJRY56Z1+XBEb05rUZ8H3vueW9/4lszsfX7HEhERqRBUNAO2ZeYA0FAjzUREREQkzNStUZXx1yQx6tzWfPz9L1w4dj4rf87wO5aIiEjYU9EMSMs4WDTTSDMRERERCT8REcbwM07mzRt6sGfvAS5+9kumpGzyO5aIiEhYU9EMSM/IBtDlmSIiIiIS1rrF1+PDEb3p0qwud0/7jrunLSN73wG/Y4mIiIQlFc3IewgAoAcBiIiIiEjYaxBTjdf/1J0RZ57ClJTNDBj3JT9u/83vWCIiImFHRTMgLTOHqEijbvWqfkcRERERESmzyAjjjn6teOWPXfklI5sLx87no+Vb/Y4lIiLlYNeuXTz77LOlWve8885j165dAU507K6//npWrlxZ5PIHHniA2bNnBzFRHhXNyBtp1jAmmogI8zuKiIiIiEjA/KFVQz4c0ZtTGtbkpklLeOj9lezdn+t3LBERCaDiimb79+8vdt0ZM2ZQp06dgOY52j4LM378eNq2bVvk8oceeoizzjqrLLFKRUUzYHBSE0ad29rvGCIiIiIiAdeoznFMGdaToafGMeHLH7n8xa/4edfvfscSEZEAGTVqFOvXrychIYG77rqLuXPQI5X0AAAgAElEQVTn0rt3b5KTkw8VogYMGECXLl1o164dL7744qF14+Li2L59O6mpqbRp04YbbriBdu3a0a9fP37//ci+YujQoQwfPpykpCRatmzJBx98AMCrr75KcnIyZ555Jn379uW3337juuuuo1u3biQmJvLee+8BcODAAe68807at29Px44dGTt2LAB9+vQhJSWFAwcOMHToUNq3b0+HDh148sknD+132rRpAMyZM4fExEQ6dOjAddddR05OzqHP8uCDD9K5c2c6dOjA6tWry3xsq5R5CxVA9+axfkcQERERESk3VatEMDq5HV3j6nH3tGWcP2YeT12eyBktG/gdTUSkQnn060dZvbPsxZr8WtdrzV+7/bXI5Y888ggrVqxg6dKlAMydO5clS5awYsUK4uPjAZgwYQL16tXj999/p2vXrlx66aXExh5eC1m7di1vvvkmL730EoMGDeLtt99myJAhR+wvNTWVr7/+mvXr1/OHP/yBdevWAbBkyRK+++476tWrx7333suZZ57JhAkT2LVrF926deOss87itddeIzU1laVLl1KlShV27tx52LaXLl3Kli1bWLFiBcARl45mZ2czdOhQ5syZQ8uWLbnmmmt47rnnGDlyJAD169dnyZIlPPvsszz22GOMHz/+WA71ETTSTERERESkkji/44lMv+00GsZEM/SVr3nykx84kOv8jiUiIgHWrVu3QwUzgDFjxtCpUyd69OjBpk2bWLt27RHrxMfHk5CQAECXLl1ITU0tdNuDBg0iIiKCFi1a0Lx580Mjus4++2zq1asHwKxZs3jkkUdISEigT58+ZGdn89NPPzF79myGDRtGlSp5Y7gOtj+oefPmbNiwgdtuu42PP/6YWrVqHbZ8zZo1xMfH07JlSwCuvfZavvjii0PLL7nkkqPmPxYaaSYiIiIiUomc3KAm797Si7+9u4Kn56xl8cZfefryBGJr6knyIiJlVdyIsGCqUaPGofdz585l9uzZfPXVV1SvXv1QEaugatX+1w9ERkYWenkmgJkVOp1/n8453n77bVq1anVMuevWrcuyZcuYOXMmzz//PFOmTGHChAklXv/gZ4iMjCzVvdUK0kgzEREREZFK5riqkTw2sCOPXtqBr1N3cv6Y+aSk7jz6iiIiEnJiYmLIzMwscvnu3bupW7cu1atXZ/Xq1SxcuLBM+5s6dSq5ubmsX7+eDRs2FFoYO+eccxg7dizO5Y1m/vbbb4G80WgvvPDCoYJWwcszt2/fTm5uLpdeeikPP/wwS5YsOWx5q1atSE1NPXRJ6Ouvv84ZZ5xRps9THBXNREREREQqITNjcNemvHPzqVSLiuDyFxcyft6GQyc4IiISHmJjY+nVqxft27fnrrvuOmJ5//792b9/P23atGHUqFH06NGjTPtr2rQp3bp149xzz+X5558nOjr6iDb3338/+/bto2PHjrRr1477778fgOuvv56mTZvSsWNHOnXqxBtvvHHYelu2bKFPnz4kJCQwZMgQ/vnPfx62PDo6mldeeYWBAwfSoUMHIiIiGD58eJk+T3GsonSKSUlJLiUlxe8YIiIVkpktds4l+Z0jlKjfEZGKJCN7H3dNXcbM79M4p93x/HtgJ2pFR/mWR/3OkdTviISuVatW0aZNG79jBMXQoUO54IILuOyyy/yOUiqFfa2K63M00kxEREREpJKrFR3F80O68Lfz2zB7VTrJY+ezamuG37FERER8paKZiIiIiIhgZlzfuzlv3diDPXsPcPGzX/LOt5v9jiUiIiHk1VdfDdtRZqWhopmIiIiIiBzSNa4eH4w4jU6N6/Dnycv427vLydl/wO9YIiIiQaeimYiIiIiIHKZhTDSTru/OsNOb85+FPzHohYVs2fW737FERESCSkUzERERERE5QpXICO45rw3PD+nM+vQsLhgzjy9+2OZ3LBERkaBR0UxERERERIrUv/2JTL+1Fw1jorn2la8ZO2ctubnO71giIiLlTkUzEREREREpVvMGNXnnllNJ7nQSj3/yA9e/lsLuPfv8jiUiIqVUs2bNoO0rJSWFESNGFLn8559/DtmHC6hoJiIiIiIiR1W9ahWeGpzA3y9qx7y12zh/7DxWbNntdywREQmyAweO7eEwSUlJjBkzpsjlJ510EtOmTStrrHKhopmIiIiIiJSImXF1zzgmD+vJgVzHJc8tYPI3P/kdS0SkUhs1ahTjxo07ND169Ggee+wxsrKy6Nu3L507d6ZDhw689957xW4nNTWV1q1bc9VVV9GmTRsuu+wy9uzZA0BcXBx//etf6dy5M1OnTmXWrFn07NmTzp07M3DgQLKysgD45ptvOPXUU+nUqRPdunUjMzOTuXPncsEFFwDw+eefk5CQQEJCAomJiWRmZpKamkr79u0ByM7O5o9//CMdOnQgMTGRzz77DIBXX32VSy65hP79+9OiRQvuvvvugB/HwlQJyl5ERERERKTC6Ny0Lh/cdhq3v7WUv769nCUbd/H/LmpHdFSk39FERHz1yz/+Qc6q1QHdZrU2rTnh3nuLXD548GBGjhzJLbfcAsCUKVOYOXMm0dHRvPPOO9SqVYvt27fTo0cPkpOTMbMit7VmzRpefvllevXqxXXXXcezzz7LnXfeCUBsbCxLlixh+/btXHLJJcyePZsaNWrw6KOP8sQTTzBq1CgGDx7M5MmT6dq1KxkZGRx33HGHbf+xxx5j3Lhx9OrVi6ysLKKjow9bPm7cOMyM5cuXs3r1avr168cPP/wAwNKlS/n222+pVq0arVq14rbbbqNJkyalOqYl5ctIMzPrb2ZrzGydmY0qZPlQM9tmZku91/V+5BQRERERkcLF1qzGxOu6cesfTmFyyiYue34Bm3bu8TuW70pwrlPNzCZ7yxeZWVzwU4pIRZKYmEh6ejo///wzy5Yto27dujRp0gTnHPfeey8dO3bkrLPOYsuWLaSlpRW7rSZNmtCrVy8AhgwZwvz58w8tGzx4MAALFy5k5cqV9OrVi4SEBCZOnMjGjRtZs2YNJ554Il27dgWgVq1aVKly+FitXr16cccddzBmzBh27dp1xPL58+czZMgQAFq3bk2zZs0OFc369u1L7dq1iY6Opm3btmzcuLEMR61kgj7SzMwigXHA2cBm4Bszm+6cW1mg6WTn3K3BziciIiIiIiUTGWHceU4rEprU4c9TlnL+mHk8dXkCZ7Y+3u9ovijhuc6fgF+dc6eY2eXAo8Dg4KcVkfJQ3Iiw8jRw4ECmTZvGL7/8cqi4NWnSJLZt28bixYuJiooiLi6O7OzsYrdTcBRa/ukaNWoA4Jzj7LPP5s033zys7fLly4+ac9SoUZx//vnMmDGDXr16HRoRVxLVqlU79D4yMpL9+/eXaL2y8OPyzG7AOufcBgAzewu4CChYNAuej0bBL0f/4oqIhK0TOsC5j/idQkREKqiz2h7Ph7f1Zvh/FnPdqymMOPMU/nx2y2IvAaqgSnKucxEw2ns/DXjGzMw558oj0C83nk/Ojz+Xx6ZFxLP/vsfJiSyXH+ESu7h7Ejff/yDbf/2VT16fSM4PK9mxdg2xVauQ++NaZi5cxMaNG8nZsJacvXvA5ZLzw+FlmJzNW/jpp5/4fPIb9EhM4PXnnqVHqxbk/LASt38fOevWkLMzncQGdbn587ms/OQjTm7WjN/27OHntHTiGp3Ezz/9xJfTJpPUsQOZWb9xXHQ19m5KJfe3LHJ+WMn6n36iZdOmtLz4QhZ9Nofln35Cx9atcXtzyPlhJT1bt+C1Z8fRq/EJrP0xlY0b1hMXkcuiX7ZwYNfOQ5lzf8ti76bUQ9MRVaOIimsR8OPqx+WZjYBN+aY3e/MKutTMvjOzaWZW6EWqZnajmaWYWcq2bdvKI6uIiIiIiJRA09jq/PfmUxnYpTG/ZGRXxoIZlOxc51Ab59x+YDcQW9jGdL4jIiXVtsUpZP72Gyc1bMiJDRsAcPmFF7Bkxfd0uXAAk96bTqvmzY+6nZbx8Tw/6U06nXshuzJ2c+MVRw6EbVCvHi/98/+45o67SLrwYs4YfBVrNmygatWq/OfJx7jj4X/QNflizr/uerJzcg5b95mJr9P5gotIuvBiqlSpwjmn9z5s+bArryDX5dLlwgEM+fNfeOmf/0e1qlXLcGTKxsrpDxpF79DsMqC/c+56b/pqoHv+SzHNLBbIcs7lmNkwYLBz7szitpuUlORSUlLKM7qISKVlZoudc0l+5wgl6ndERArnnONArqNKZOn/Ph+u/U4Jz3VWeG02e9PrvTbbi9u2+h2R0LVq1SratGnjd4wyS01N5YILLmDFihV+Ryk3hX2tiutz/BhptgXIP3KssTfvEOfcDufcwXLkeKBLkLKJiIiIiEgZmFmZCmZh7qjnOvnbmFkVoDawIyjpRETkmPjRm30DtDCzeDOrClwOTM/fwMxOzDeZDKwKYj4REREREZHSOOq5jjd9rff+MuDT8rqfmYjIsYiLi6vQo8xKI+hFM++6/VuBmeQVw6Y45743s4fMLNlrNsLMvjezZcAIYGiwc4qISPgxs/5mtsbM1pnZqGLaXWpmzsySvOk4M/vdzJZ6r+eDl1pERCqKEp7rvAzEmtk64A6gyP5KRMKHat+hrzRfIz+enolzbgYwo8C8B/K9vwe4J9i5REQkfJlZJDAOOJu8Gy9/Y2bTnXMrC7SLAW4HFhXYxHrnXEJQwoqISIVVgnOdbGBgsHOJSPmJjo5mx44dxMbGVtaHoIQ85xw7duwgOjr6mNbzpWgmIiJSDroB65xzGwDM7C3gImBlgXZ/Bx4F7gpuPBERERGpiBo3bszmzZvRU25DW3R0NI0bNz6mdVQ0ExGRiqIRsCnf9Gage/4GZtYZaOKc+9DMChbN4s3sWyAD+Jtzbl7BHZjZjcCNAE2bNg1kdhEREREJU1FRUcTHx/sdQ8pBpX2sjYiIVC5mFgE8AfylkMVbgabOuUTy7i/zhpnVKtjIOfeicy7JOZfUoEGD8g0sIiIiIiK+UtFMREQqii1Ak3zTjb15B8UA7YG5ZpYK9ACmm1mScy7HObcDwDm3GFgPtAxKahERERERCUkqmomISEXxDdDCzOLNrCpwOTD94ELn3G7nXH3nXJxzLg5YCCQ751LMrIH3IAHMrDnQAtgQ/I8gIiIiIiKhosLc02zx4sXbzWxjGTZRH9geqDzlJBwyQnjkVMbAUMbACfWczfwOcDTOuf1mdiswE4gEJjjnvjezh4AU59z0YlY/HXjIzPYBucBw59zO4vanfidkKGPghENOZQyMcMgY8v1OsJWx3wmHr3k4ZITwyKmMgREOGSE8coZ6xiL7HHPOBTNIyDKzFOdckt85ihMOGSE8cipjYChj4IRLTgmccPiaK2NghENGCI+cyhgY4ZBRAiscvubhkBHCI6cyBkY4ZITwyBkOGYuiyzNFREREREREREQKUNFMRERERERERESkABXN/udFvwOUQDhkhPDIqYyBoYyBEy45JXDC4WuujIERDhkhPHIqY2CEQ0YJrHD4modDRgiPnMoYGOGQEcIjZzhkLJTuaSYiIiIiIiIiIlKARpqJiIiIiIiIiIgUoKKZiIiIiIiIiIhIAZWuaGZm/c1sjZmtM7NRhSyvZmaTveWLzCwuyPmamNlnZrbSzL43s9sLadPHzHab2VLv9UAwM3oZUs1subf/lEKWm5mN8Y7jd2bW2YeMrfIdo6VmlmFmIwu0CfqxNLMJZpZuZivyzatnZp+Y2Vrv37pFrHut12atmV0b5Iz/NrPV3tfzHTOrU8S6xX5vlHPG0Wa2Jd/X87wi1i3290AQck7OlzHVzJYWsW5QjqWUn1Dvc7wM6ncCky8k+xxvv+p3yi9jSPU76nMk1PudcOlzvBzqd0qXS31O+eZUvxNszrlK8wIigfVAc6AqsAxoW6DNzcDz3vvLgclBzngi0Nl7HwP8UEjGPsAHPh/LVKB+McvPAz4CDOgBLAqBr/0vQDO/jyVwOtAZWJFv3r+AUd77UcCjhaxXD9jg/VvXe183iBn7AVW8948WlrEk3xvlnHE0cGcJvheK/T1Q3jkLLH8ceMDPY6lXuX3tQ77P8farfqd8vvYh0ed4+1W/U34ZQ6rfUZ9TuV/h0O+ES5/j5VC/U7os6nPKN6f6nSC/KttIs27AOufcBufcXuAt4KICbS4CJnrvpwF9zcyCFdA5t9U5t8R7nwmsAhoFa/8BdBHwmsuzEKhjZif6mKcvsN45t9HHDAA4574AdhaYnf/7biIwoJBVzwE+cc7tdM79CnwC9A9WRufcLOfcfm9yIdC4PPZdUkUcx5Ioye+BgCkup/e7ZRDwZnntX3wV8n0OqN8pJyHT54D6nUAJh35HfU6lF/L9TgXqc0D9TqHU5wSO+p3QUNmKZo2ATfmmN3PkL+lDbbwfmt1AbFDSFeANl04EFhWyuKeZLTOzj8ysXVCD5XHALDNbbGY3FrK8JMc6mC6n6B9Wv48lwPHOua3e+1+A4wtpE0rH9Dry/rJWmKN9b5S3W71h1ROKGPodSsexN5DmnFtbxHK/j6WUTVj1OaB+J4BCvc8B9TuBFC79jvqcii+s+p0Q73NA/U4gqc8JLPU7QVTZimZhw8xqAm8DI51zGQUWLyFv6G0nYCzwbrDzAac55zoD5wK3mNnpPmQoETOrCiQDUwtZHArH8jAub6yq8ztHUczsPmA/MKmIJn5+bzwHnAwkAFvJGw4cyq6g+L+8hM3PmYQ/9TuBEW59DqjfKaNw6nfU50jICIM+B8LkZyLc+h31OWWmfifIKlvRbAvQJN90Y29eoW3MrApQG9gRlHQeM4sirxOZ5Jz7b8HlzrkM51yW934GEGVm9YOZ0Tm3xfs3HXiHvCGg+ZXkWAfLucAS51xawQWhcCw9aQeHc3v/phfSxvdjamZDgQuAq7wO7wgl+N4oN865NOfcAedcLvBSEfv2/TjCod8vlwCTi2rj57GUgAiLPsfbt/qdwAmHPgfU7wREuPQ76nMqjbDod8Khz/H2rX4ncNTnBIj6neCrbEWzb4AWZhbvVeQvB6YXaDMdOPikjsuAT4v6gSkP3nW/LwOrnHNPFNHmhIP3HjCzbuR9HYPW2ZlZDTOLOfievJsmrijQbDpwjeXpAezONyQ32IqscPt9LPPJ/313LfBeIW1mAv3MrK43DLefNy8ozKw/cDeQ7JzbU0SbknxvlGfG/PeRuLiIfZfk90AwnAWsds5tLmyh38dSAiLk+xxQv1MOwqHPAfU7gcoYLv2O+pzKIeT7nXDoc7z9qt8JLPU5AaJ+xwcuBJ5GEMwXeU85+YG8p0nc5817iLwfDoBo8oa2rgO+BpoHOd9p5A1X/Q5Y6r3OA4YDw702twLfk/cUjIXAqUHO2Nzb9zIvx8HjmD+jAeO847wcSPLp612DvI6hdr55vh5L8jq1rcA+8q4v/xN595KYA6wFZgP1vLZJwPh8617nfW+uA/4Y5IzryLs2/uD35cEnL50EzCjueyOIGV/3vt++I69jOLFgRm/6iN8DwczpzX/14Pdhvra+HEu9yvXrH9J9jpdB/U7gcoZcn+PtV/1O+WUMqX6nsIze/FdRn1MpXoV9rxFC/Q5h0Od4GdTvlD6T+pzyzal+J8gv8wKLiIiIiIiIiIiIp7JdnikiIiIiIiIiInJUKpqJiIiIiIiIiIgUoKKZiIiIiIiIiIhIASqaiYiIiIiIiIiIFKCimYiIiIiIiIiISAEqmomEKDPrY2Yf+J1DREQqB/U7IiISLOpzJFyoaCYiIiIiIiIiIlKAimYiZWRmQ8zsazNbamYvmFmkmWWZ2ZNm9r2ZzTGzBl7bBDNbaGbfmdk7ZlbXm3+Kmc02s2VmtsTMTvY2X9PMppnZajObZGbm2wcVEZGQoH5HRESCRX2OVHYqmomUgZm1AQYDvZxzCcAB4CqgBpDinGsHfA486K3yGvBX51xHYHm++ZOAcc65TsCpwFZvfiIwEmgLNAd6lfuHEhGRkKV+R0REgkV9jghU8TuASJjrC3QBvvH+MHIckA7kApO9Nv8B/mtmtYE6zrnPvfkTgalmFgM0cs69A+Ccywbwtve1c26zN70UiAPml//HEhGREKV+R0REgkV9jlR6KpqJlI0BE51z9xw20+z+Au1cKbefk+/9AfQzKyJS2anfERGRYFGfI5WeLs8UKZs5wGVm1hDAzOqZWTPyfrYu89pcCcx3zu0GfjWz3t78q4HPnXOZwGYzG+Bto5qZVQ/qpxARkXChfkdERIJFfY5UeqrkipSBc26lmf0NmGVmEcA+4BbgN6CbtyydvHsBAFwLPO91FBuAP3rzrwZeMLOHvG0MDOLHEBGRMKF+R0REgkV9jgiYc6UdSSkiRTGzLOdcTb9ziIhI5aB+R0REgkV9jlQmujxTRERERERERESkAI00ExERERERERERKUAjzURERERERERERApQ0UxERERERERERKQAFc1EimBmr5rZwyVsm2pmZ5V3JhERkcIEqs86lu2IiEjwhds5ipldZWazStDueTO7PxiZyoOZ9TGzzfmmfT/2EhhV/A4gIiIiIiIiIhWPc24SMKkE7YYHcr9m9gKw2Dn3YiC3K5WPRpqJVHBmpuK4iIiIiIiUSpieT5wLzCg4M0w/i/hIRTMJa96w17vM7Dsz+83MXjaz483sIzPLNLPZZlY3X/tkM/vezHaZ2Vwza5NvWaKZLfHWmwxEF9jXBWa21Ft3gZl1LGHG883sWzPLMLNNZja6wPLTvO3t8pYP9eYfZ2aPm9lGM9ttZvO9eYcN/c13HM7y3o82s2lm9h8zywCGmlk3M/vK28dWM3vGzKrmW7+dmX1iZjvNLM3M7jWzE8xsj5nF5mvX2cy2mVlUST67iIj8Tzj0WYVkvsHM1nn9w3QzO8mbb2b2pJmle/3bcjNr7y07z8xWetm2mNmdpTpgIiJhKhx+31veZZ7Pe+cAmWb2uZk1y7fcmdktZrYWWHu0fZlZEzP7r3eusMPMnvHmDzWz+d774vqOwy47Lar/yZdtuJmt9bKMMzPLt7wjsMs5t9nb/5fefncAo82smpk9ZmY/eec+z5vZcfnWv8j7nBlmtt7M+nvz/2hmq7zjtcHMhpXkWEt4U9FMKoJLgbOBlsCFwEfAvUAD8r7HRwCYWUvgTWCkt2wG8L6ZVbW8AtK7wOtAPWCqt128dROBCcAwIBZ4AZhuZtVKkO834BqgDnA+cJOZDfC228zLO9bLlAAs9dZ7DOgCnOpluhvILeExuQiY5u1zEnAA+DNQH+gJ9AVu9jLEALOBj4GTgFOAOc65X4C5wKB8270aeMs5t6+EOURE5HCh3mcdYmZnAv8krx84EdgIvOUt7gec7n2O2l6bHd6yl4FhzrkYoD3w6bHsV0SkggiH3/dXAX8n7xxhKUdeRjkA6A60LW5fZhYJfEBePxEHNOJ//UV+xfUdhxyl/znoAqAr0NFrd06+ZecBH+ab7g5sAI4H/g94xMuQQN65TyPgAW/f3YDXgLvIO5c6HUj1tpPu7bcW8EfgSTPrXMjnlApERTOpCMY659Kcc1uAecAi59y3zrls4B0g0Ws3GPjQOff/2bvzeKvqev/jrw+DIIiA4MggmDhnmjildbUcKFO6Wc6llXK7ZVr35s1+lZppabfRqw1mmpVjWoaFcw6ZE2imghOZysEJEBCUgyCf3x9roRs8wAHOZp19zuv5eOzHWcN3rfXe8HjstfZnf9d33VQWfb4HrE1RlNoN6A78KDMXZOZVwPiaY4wBfp6Z92bmG5l5MTC/3G65MvO2zHw4Mxdl5kMUJ8V/K1cfAdycmZeVx52RmQ9GRBfg08CJmTm1POZdmTm/lf8md2fmNeUx52Xm/Zl5T2YuzMynKU5yizN8GHghM7+fmc2ZOScz7y3XXQwcBVCeDA+nOGlLklZNuz5nLeVI4MLMfKA8/3wV2D0ihgELgD7AVkBk5qOZ+Xy53QKKL1jrZubMzHxgJY8rSR1BI3ze/zkz7yg/479G8Rk/pGb9dzLz5cyct4Jj7ULx4/tJmflq+Z3izhaOt7xzR63lnX8WOyszZ2Xms8CtFAWwxQ5gyVszn8vM/8vMhUBz+V6+VL63OcC3gcPKtp8pj31T+V1qamY+BpCZf87Mf2bhduBG4L3L/NdVh2DRTB3BizXT81qYX6ec3oTiVwoAMnMRMIXil4VNgKmZmTXbPlMzvSnw32X331kRMQsYUm63XBGxa0TcWnZVng18luLXHMp9/LOFzQZSdL1uaV1rTFkqwxYR8aeIeCGKWza/3YoMAH+k+OIznOKXstmZed8qZpIktfNz1lKWzjCXokfAoMz8C3AucB7wUkScHxHrlk0PpviV/5nydp/dV/K4ktQRNMLn/ZvfGcrP+JeX2rb2O8XyjjUEeKYsSi3TCs4dtZZ5/qlp80LN9GuU/54R0Y+iKHfXMt7H+kAv4P6a93F9uRyW890oIj4YEfeUt4zOojjXDWyprToOi2bqTJ6j+LAHinvqKT4UpwLPA4Nq74UHhtZMTwHOzMx+Na9emXlZK457KTAWGJKZfYGfAYuPMwV4RwvbTKf4FaSlda9SfNAvfh9deetDfrFcav6nwGPAiMxcl6JreG2GzVoKXv4SdiVFb7NPYC8zSVpTqjpnLS9Db4pbcqYCZOY5mbkTsA3FbS4nlcvHZ+ZoYAOK24quXMnjSlJnUuXn/Zu9yiJiHYpbQJ+rWV/7nWJ5x5oCDI1WDLK/rHPHUpZ7/lmB/YG/ZOYby3gf0ymKltvWvI++mbm4iNni97PylterKXoCbpiZ/Sh6s8XSbdWxWDRTZ3IlcEBEfCCKgez/m6JL8V3A3cBC4ISI6B4RH6XoZrzYL4DPlr3GIiJ6RzHAf59WHLcP8HJmNpf3yB9Rs+4SYJ+IOCQiukXEgIjYofyF6ULgBxGxSUR0jYjdyw/rJ4Ce5fG7A18HVkE76/YAACAASURBVDRuQR/gFWBuRGwF/GfNuj8BG0fEF8sxCfpExK41638NHAMchEUzSVpTqjpn1boM+FRE7FCef75NcXvR0xGxc7n/7hQ/5jQDi8oxeI6MiL7lbUav0PrxOCWpM6ry8/5DUTyUbC2Ksc3uycwpy2i7vGPdR1HgO6tc3jMi9lh6B8s6d7RwrGWef1rznlhyPLMllN+zfkExHtkGZa5BEbF4TLRflsf+QER0KddtBaxF8Z1rGrAwIj5IMUabOjiLZuo0MvNxih5T/0fxC8OBwIGZ+Xpmvg58lKI49DLF2AK/r9l2AnAcRXfimcDksm1rfA44PSLmUAww+eYv7uU9+B+iODm+TDEA57vK1V8GHqYYt+Bl4GygS2bOLvd5AcWvLa8CSzxNswVfpijWzaE4SVxRk2EOxa2XB1J0c34S2Ltm/d8oTmYPZGZtd3BJUp1UeM6qzXAz8A2KX9afp/jlffGYL+tSnE9mUtxCMwP433LdJ4CnoxgO4LMUY9NIklpQ8ef9pcCp5b53KnMsK+cyj1X26jqQYlD9Zym+mxzawm6Wd+6oPdbyzj/LVPbI25/idsvl+UqZ/57yXHUzsGV57PsoB/kHZgO3A5uW35lOoPguN5Piu9XYFWVS44slb4+WpLeLiL8Al2bmBVVnkSRJkrR6IuJXQFNmfr3qLG2lvKvn3MzcZYWNpVZa4T3Hkjq3iNgZeDcwuuoskiRJkrQcp1YdQB2LRTNJyxQRFwMfAU4suyRLkiRJUrtT3loptSlvz5QkSZIkSZKW4oMAJEmSJEmSpKV0mNszBw4cmMOGDas6hiR1SPfff//0zFy/6hztiecdSaofzztv53lHkupjeeecDlM0GzZsGBMmTKg6hiR1SBHxTNUZ2hvPO5JUP5533s7zjiTVx/LOOd6eKUmSJEmSJC3FopkkSZIkSZK0FItmkiRJkiRJ0lI6zJhmLVmwYAFNTU00NzdXHaXuevbsyeDBg+nevXvVUSRJkrQavIaVJKl96NBFs6amJvr06cOwYcOIiKrj1E1mMmPGDJqamhg+fHjVcSRJkrQavIaVJKl96NC3ZzY3NzNgwIAOfbEBEBEMGDCgU/waKUmS1NF5DStJUvvQoYtmQIe/2Fiss7xPSZKkzqCzXNt1lvcpSWpMHb5oJkmSJEmSJK2sDj2mWWs9N2se8xa8UZd9vzJ7FmOv/h1Hffq4ldruM4cfzA9/9kvW7duv1dtMmzOf035+98pGlNQJbLPJupx64LZVx5AkLXoDFjbDwvnl39rp8u+CvtA8GzKBbOEvLS9/27pyGpY/nzXLy7+zZr3CpVeP5XOfPrImfC7xp2bizfkPHT6GS3/6PfqtvyH023QV/5EkSWofLJrV2SuzZ3PJr37xtqLZwoUL6dZt2f/8v7zs6npHkyRJUksyi+LV/Lkw/xV4fW45PaecfqWYf71ctvj1+lxYMK94LV0IW/x30YIVH3//K+HlVQkeEFH8hXKalZh/a3rWrFn85KJL+dynj6hpEzXXsLHEposnxl1xUTHbxadhSpIaX6cpmn3z2olMeu6VNt1na3pufO0L32LK0//i4H33pHv37vTs2ZP+/fvz2GOP8cQTT/CRj3yEKVOm0NzczIknnsiYMWMAGDZsGBMmTGDu3Ll88IMfZM899+Suu+5i0KBB/PGPf2Tttdd+27Fen96DK/5jhzZ9j5IkSQ0tsyhovTYdXp1R/p1e83cGvDqtnH65LIjNgWzNXQgBa60DPfpAj3WK6bV6Q68B0K0HdOtZvnos+bf7MpZ36wFde8Cc3jBwBETwzXFPMun5uW8d721DgK38mGCtuYY9+Qun8s+np7DDPoes2jXsy3P54K5bt+oaVpKk9qrTFM2qctZZZ/HII4/w4IMPctttt3HAAQfwyCOPvPlY7QsvvJD11luPefPmsfPOO3PwwQczYMCAJfbx5JNPctlll/GLX/yCQw45hKuvvpqjjjqqircjSZLUPmQWRa5Zz8CsZ4vXnOdbKIhNhzfmt7yPbmtD74Hla31YfyvouW5RBHuzGNZnycJYj3XL+XWge2/oUochgh99tCi+AXTpBrHmhyH2GlaSpE5UNGsvY/nssssub15sAJxzzjn84Q9/AGDKlCk8+eSTb7vgGD58ODvsUPQg22mnnXj66afXWF5JkqRKZBZFr9qi2BKvKbDg1SW36d77rSJYn41ho3cWvb4WF8V6DYTeA8q/A98qTLVjXsNKklSdTlM0ay96937r4uy2227j5ptv5u6776ZXr17stddeNDc3v22bHj16vDndtWtX5s2bt0aySpIk1VVm0TvsxUnw0iSY+TTMnvJWYWzBa0u279kP+g2FAZvDO95fTC9+9R0Ca7f+AUpaOV7DSpI6I4tmddanTx/mzJnT4rrZs2fTv39/evXqxWOPPcY999yzhtNJkiStIa+/BtMehRcn1rwegXkz32qzRFHsA0sWxfoNgZ59q8vfyXgNK0mSRbO6GzBgAHvssQfbbbcda6+9NhtuuOGb60aNGsXPfvYztt56a7bcckt22223CpNKkiS1gUWLilsqawtjL06El58CsmjTvTdssDVsfRBsuB1suG0x32u9SqPrLV7DSpIEkZlVZ2gTI0eOzAkTJiyx7NFHH2XrrbeuKNGa19ner6Q1JyLuz8yRVedoT1o670idTmZRDHv6TnjugbJINqlmrLGA9YYXRbHFxbENt4V+w+ozgH4H0dmu6Vp6v5533s7zjiTVx/LOOfY0kyRJUutkwrTH4Zm/Fa+n/wZzXyjW9exXFMZ2POqtItkGWzXEYPuSJEktsWgmSZKkli1aVAzQ/8zfit5kz9wFr00v1vXZGIbtCcP2gE33hIEjIKLavJIkSW3IopkkSZIKi96AFx4qimNP/w2eveutgfr7DoHN9ymLZHvAeptZJJMkSR2aRTNJkqTObPpkeHwcPP1XePYemP9Ksbz/cNjqgKJAtuke0H/TanNKkiStYRbNJEmSOpPM4omWj15bvF6aVCwfMAK2+2hxq+Wm74G+g6rNKUmSVDGLZpIkSR3dokUw9X54dGxRKJv5LyCK4tios4seZf2GVJ1SkiSpXbFo1s6ss846zJ07t+oYkiSp0b2xsBiT7NFr4dE/wZznoEt32OzfYM8vwpYfgnU2qDqlOgivYSVJHZFFM0mSpI5i4Xx46vaiR9nj4+C1GdBtbdj8A7D1abDF/rB2v6pTSpIkNYTOUzS77mR44eG23edG74QPnrXcJieffDJDhgzh85//PACnnXYa3bp149Zbb2XmzJksWLCAM844g9GjR7dtNkmS1Dm8/hpMvqnoUfbEDcVA/j3WLQpkWx9UFMzW6l11Sq0qr2ElSapM5ymaVeTQQw/li1/84psXHFdeeSU33HADJ5xwAuuuuy7Tp09nt91246CDDiJ8bLskSWqtFyfC/b+Cf1wB82dDrwGw7UeKQtnw90G3HlUnVAPzGlaSpM5UNFvBr2n1suOOO/LSSy/x3HPPMW3aNPr3789GG23El770Je644w66dOnC1KlTefHFF9loo40qyShJkhrE66/BxD8UxbKm+6BrD9hmNOx4FGy6B3TtPJd2nYbXsJIkVcYrqzXg4x//OFdddRUvvPAChx56KJdccgnTpk3j/vvvp3v37gwbNozm5uaqY0qSpPbqxUlw/0Vv9SobuAXs/2141+HQa72q06mD8hp2+SJiFPBjoCtwQWaetdT6HsCvgZ2AGcChmfl0zfqhwCTgtMz83prKLUlqPYtma8Chhx7Kcccdx/Tp07n99tu58sor2WCDDejevTu33norzzzzTNURJUlSe7Ng3lu9yqbcC13XKnqV7fQp2PQ94C1xqjOvYZctIroC5wH7Ak3A+IgYm5mTapp9BpiZmZtHxGHA2cChNet/AFy3pjJLklaeRbM1YNttt2XOnDkMGjSIjTfemCOPPJIDDzyQd77znYwcOZKtttqq6oiSJKm9eOlRmHARPHQ5NM+GASNgvzOLXmW9B1SdTp2I17DLtQswOTOfAoiIy4HRFD3HFhsNnFZOXwWcGxGRmRkRHwH+Bby65iJLklaWRbM15OGH33rq0cCBA7n77rtbbDd37tw1FUmSJLUXC+bBxGuKWzAX9yrb+iAY+alirDJ7lakiXsMu0yBgSs18E7Drstpk5sKImA0MiIhm4CsUvdS+vLyDRMQYYAzA0KFD2ya5JKnVLJpJkiRVZdazcPdP4B+Xlr3KNof9zoB3HWGvMqnjOg34YWbOXdGTRzPzfOB8gJEjR2b9o0mSatW1aNaKwTGPAf4XmFouOjczL6hZvy5FF+drMvP4emaVJElaY6Y9Dnf+CB6+EgjY5qBirLJhe9qrTGoMU4EhNfODees7zdJtmiKiG9CX4oEAuwIfi4jvAv2ARRHRnJnn1j+2JGll1K1o1srBMQGuWE5B7FvAHauTIzNZ0S84HUGmPzxJktTuPfd3+OsP4NFroVtP2GUM7H489B1UdTK1M17DtnvjgRERMZyiOHYYcMRSbcYCRwN3Ax8D/pLFG37v4gYRcRow14KZJLVP9exp1prBMZcpInYCNgSuB0auSoCePXsyY8YMBgwY0KEvOjKTGTNm0LNnz6qjSJKkljxzF9zxPfjnLdCjL7zvy7DrZ6H3wKqTqR3yGrb9K8coOx64geKumgszc2JEnA5MyMyxwC+B30TEZOBlisKaJKmB1LNo1prBMQEOjoj3AU8AX8rMKRHRBfg+cBSwz7IOsKKBMQcPHkxTUxPTpk1b5TfRKHr27MngwYOrjiFJkhbLhMk3w1+/D8/eDb0GwgdOhZ0/Az37Vp1O7ZjXsI0hM8cB45ZadkrNdDPw8RXs47S6hJMktYmqHwRwLXBZZs6PiP8ALgbeD3wOGJeZTcv7dW1FA2N2796d4cOH1yW4JElSixa9AY+OLYplLzwM6w6GD/4v7HgUrNWr6nRqAF7DSpLUPtSzaLbCwTEzc0bN7AXAd8vp3YH3RsTngHWAtSJibmaeXMe8kiRJq+6NBfDQlXDnD2HGk8WTMEefB+88BLqtVXU6SZIkraR6Fs1WODhmRGycmc+XswcBjwJk5pE1bY4BRlowkyRJ7dKCefDAb+Cuc2D2FNjonfDxX8HWB0GXrlWnkyRJ0iqqW9GslYNjnhARBwELKQbHPKZeeSRJktrUgmYYfwH87Ufw6jQYsht8+Iew+T7QgQdvlyRJ6izqOqZZKwbH/Crw1RXs41fAr+oQT5LUgCJiFPBjih9kLsjMs1pocwhwGpDAPzLziHL52cABZbNvZeYV5fIAzqAYsPkN4KeZeU6d34oa1aJF8PDv4C9nwOxnYbO94H3/A8P2qDqZJEmS2lDVDwKQJKnVIqIrcB6wL8VTmcdHxNjMnFTTZgTFDzJ7ZObMiNigXH4A8G5gB6AHcFtEXJeZr1D0dB4CbJWZixZvI73NP/8CN50KLzwEG20PB50D79i76lSSJEmqA4tmkqRGsgswOTOfAoiIy4HRwKSaNscB52XmTIDMfKlcvg1wR2YuBBZGxEPAKOBK4D+BIzJz0VLbSIXnH4KbTy2KZn2HwkcvgO0Ohi5dqk4mSZKkOvFKT5LUSAYBU2rmm8pltbYAtoiIv0XEPeXtnAD/AEZFRK+IGAjszVtPeX4HcGhETIiI68ream8TEWPKNhOmTZvWZm9K7disZ+H3/wE/fx9MfQD2OxO+MAG2/7gFM0mSpA7OnmaSpI6mGzAC2AsYDNwREe/MzBsjYmfgLmAacDfF+GVQ3K7ZnJkjI+KjwIXAe5fecWaeD5wPMHLkyKz3G1GF5s2Ev34f7j2/mN/jRNjzS7B2v2pzSZIkaY2xaCZJaiRTeat3GBRFsalLtWkC7s3MBcC/IuIJiiLa+Mw8EzgTICIuBZ6o2eb35fQfgIvqE1/t3oJmuO98+Ov3oPkVeNfhsPf/g35DVrytJEmSOhSLZpKkRjIeGBERwymKZYcBRyzV5hrgcOCi8jbMLYCnyocI9MvMGRGxPbA9cGPNNnsD/wL+jbeKaeosFi2Ch68sn4g5BTbfB/b5Jmy0XdXJJEmSVBGLZpKkhpGZCyPieOAGoCtwYWZOjIjTgQmZObZct19ETKK4/fKkslDWE/hrRAC8AhxVPhQA4Czgkoj4EjAXOHbNvjNVavItxSD/LzwMG78LRp8Lm+1VdSpJkiRVzKKZJKmhZOY4YNxSy06pmU7gv8pXbZtmiidotrTPWcABbR5W7dtLj8H1J8NTt0I/n4gpSZKkJVk0kyRJncv8OXDbWXDvz2Ct3sUTMXc5Drr1qDqZJEmS2hGLZpIkqXPIhEeuhhu+BnNfgB0/AfucBr0HVp1MkiRJ7ZBFM0mS1PG9OAnGnQTP3Akb7wCHXQKDR1adSpIkSe2YRTNJktRxNb/y1q2YPfrAh38I7z4aunStOpkkSZLaOYtmkiSp48mEh66Em74Bc1+Cd38SPnAq9B5QdTJJkiQ1CItmkiSpY3nhkeJWzGfvgk3eDYdfBoN2qjqVJEmSGoxFM0mS1DE0z4ZbvwP3nQ89+8KBP4YdPwldulSdTJIkSQ3IopkkSWpsmfCPy+GmU+DVaTDyU/D+b0Cv9apOJkmSpAZm0UySJDWuFx6GP38ZptwDg0bCkVfCJjtWnUqSJEkdgEUzSZLUeObPgb+cCff9HNbuDwedCzsc6a2YkiRJajMWzSRJUmN5/Lqid9krU2Hkp+ED3ygKZ5IkSVIbsmgmSZIaw5wX4Lr/gUl/hA22gY9fBEN2qTqVJEmSOiiLZpIkqX1btAge+BXcdBosbC4G+X/PCdBtraqTSZIkqQOzaCZJktqvlx6Da08sBvof/j748I9gwDuqTiVJkqROwKKZJElqfxY0w1+/D3f+EHqsA6N/AjscARFVJ5MkSVInYdFMkiS1L0/fWfQumzEZtj8U9v829B5YdSpJkiR1MhbNJElS+/Day3DTKfD330C/TeGo38PmH6g6lSRJkjopi2aSJKlamfDI1XD9yUXhbI8T4d9OhrV6VZ1MkiRJnZhFM0mSVJ2Zz8Cf/xsm3wSb7Fj0Ltt4+6pTSZIkSRbNJElSBd5YCPf+DG49EwgYdRbsMga6dK06mSRJkgRYNJMkSWvaC4/A2OPhub/DiP3hgO9DvyFVp5IkSZKWYNFMkiStGQvnwx3fgzt/AD37wccuhG0/ChFVJ5MkSZLexqKZJEmqv2fvhbFfgOmPw/aHwajvQK/1qk4lSZIkLVOXqgNIkqQObP5cuO4rcOH+8PqrcORV8NGfWzCT1PAiYlREPB4RkyPi5BbW94iIK8r190bEsHL5vhFxf0Q8XP59/5rOLklqHXuaSZKk+ph8C1z7RZj9bDHI/wdOgR59qk4lSastIroC5wH7Ak3A+IgYm5mTapp9BpiZmZtHxGHA2cChwHTgwMx8LiK2A24ABq3ZdyBJag2LZpIkqW299jLc8DX4x6UwYAR86nrYdPeqU0lSW9oFmJyZTwFExOXAaKC2aDYaOK2cvgo4NyIiM/9e02YisHZE9MjM+fWPLUlaGRbNJElS28iESdfAuJOKwtl7vwzvOwm696w6mSS1tUHAlJr5JmDXZbXJzIURMRsYQNHTbLGDgQeWVTCLiDHAGIChQ4e2TXJJUqtZNJMkSavvledh3JfhsT/Bxu+Co34PG29fdSpJarciYluKWzb3W1abzDwfOB9g5MiRuYaiSZJKFs0kSdKqy4QHfg03fgPemA/7ng67fR66eokhqUObCgypmR9cLmupTVNEdAP6AjMAImIw8Afgk5n5z/rHlSStCq9oJUnSqnn5Kbj2RPjXHbDpnnDQOTDgHVWnkqQ1YTwwIiKGUxTHDgOOWKrNWOBo4G7gY8BfMjMjoh/wZ+DkzPzbGswsSVpJFs0kSdLKeWMh3Psz+MsZ0KUbfPiH8O5joEuXqpNJ0hpRjlF2PMWTL7sCF2bmxIg4HZiQmWOBXwK/iYjJwMsUhTWA44HNgVMi4pRy2X6Z+dKafReSpBWxaCZJklrvhYdh7Bfgub/DFqPggB9A30FVp5KkNS4zxwHjllp2Ss10M/DxFrY7Azij7gElSautrj8JR8SoiHg8IiZHxMktrD8mIqZFxIPl69hy+aYR8UC5bGJEfLaeOSVJ0gosaIZbTofz94LZTfCxi+Dwyy2YSZIkqcOqW0+ziOgKnAfsS/EI5vERMTYzJy3V9IrMPH6pZc8Du2fm/IhYB3ik3Pa5euWVJEnL8PSdxdhlMybDDkfCfmdAr/WqTiVJkiTVVT1vz9wFmJyZTwFExOXAaGDpotnbZObrNbM9qHOPOEmS1ILm2XDTKXD/r6DfpvCJa+Ade1edSpIkSVoj6lmMGgRMqZlvKpct7eCIeCgiroqINx/bHBFDIuKhch9nt9TLLCLGRMSEiJgwbdq0ts4vSVLn9eif4Nxd4IFfw+7Hw+futmAmSZKkTqXqHlzXAsMyc3vgJuDixSsyc0q5fHPg6IjYcOmNM/P8zByZmSPXX3/9NRZakqQOa86LcOUn4YojofdAOPYW2P9MWKt31ckkSZKkNaqet2dOBYbUzA8ul70pM2fUzF4AfHfpnWTmcxHxCPBe4Ko65JQkSZnw99/AjV8vBv3/wCnwnhOga/eqk0nSKomIOUC2tArIzFx3DUeSJDWYehbNxgMjImI4RbHsMOCI2gYRsXFmPl/OHgQ8Wi4fDMzIzHkR0R/YE/hhHbNKktR5zfhnMdD/03+FTfeAA8+BgZtXnUqSVktm9qk6gySpsdWtaJaZCyPieOAGoCtwYWZOjIjTgQmZORY4ISIOAhYCLwPHlJtvDXw/IpLil6DvZebD9coqSVKn9MZCuPtcuO070HUt+PCP4N1HQ5eqR2+QpNUXEct9zG9mvrymskiSGlM9e5qRmeOAcUstO6Vm+qvAV1vY7iZg+3pmkySpU3v+H/DH4+GFh2DLA+CA78G6m1SdSpLa0v0Ut2dGC+sS2GzNxpEkNZq6Fs0kSVI7M38O3PoduPen0Ht9OOTXsPVBEC19p5SkxpWZw6vOIElqbBbNJEnqDDLh0Wvhuq/AnOdgp0/BPqfC2v2rTiZJdVeOkzwC6Ll4WWbeUV0iSVIjsGgmSVJHN/NpGPc/8OQNsOF2cMjFMGSXqlNJ0hoREccCJwKDgQeB3YC7gfdXmUuS1P5ZNJMkqaNa+Hox0P/t34XoAvudCbt+Frp6+pfUqZwI7Azck5l7R8RWwLcrziRJagA+HkuS1FAiYlREPB4RkyPi5GW0OSQiJkXExIi4tGb52RHxSPk6tIXtzomIufXMv8Y8/Tf4+Xvhlm/CiH3g+PvgPcdbMJPUGTVnZjNARPTIzMeALSvOJElqAF45S5IaRkR0Bc4D9gWagPERMTYzJ9W0GUHxZOY9MnNmRGxQLj8AeDewA9ADuC0irsvMV8r1I4HGH+Dr1Rlw0ynw4G+h71A4/ArYclTVqSSpSk0R0Q+4BrgpImYCz1ScSZLUACyaSZIayS7A5Mx8CiAiLgdGA5Nq2hwHnJeZMwEy86Vy+TbAHZm5EFgYEQ8Bo4Ary2Lc/wJHAP++Rt5JW1u0qCiU3XRK8YTMPb8E7zsJ1upddTJJqlRmLv5cPy0ibgX6AtdXGEmS1CC8PVOS1EgGAVNq5pvKZbW2ALaIiL9FxD0Rsbib1T+AURHRKyIGAnsDQ8p1xwNjM/P55R08IsZExISImDBt2rTVfjNt5sVJcNEHYewXYP2t4LN3wj6nWTCTJCAidouIPgCZeTtwG7BjpaEkSQ3BnmaSpI6mGzAC2IviSWl3RMQ7M/PGiNgZuAuYRvHktDciYhPg42X75crM84HzAUaOHJl1Sb8yXn8Vbj8b7j4PeqwLo8+Ddx0BXfxNTJJq/JTi9vzF5rawTJKkt7FoJklqJFN5q3cYFEWxqUu1aQLuzcwFwL8i4gmKItr4zDwTOBOgfEDAExS9DTYHJkcEQK+ImJyZm9f1nayux6+DcSfB7Cmw41Gwz+nQe0DVqSSpPYrMfPOHjsxcFBF+D5IkrZAnC0lSIxkPjIiI4RTFssMoxiGrdQ1wOHBReRvmFsBT5bhl/TJzRkRsD2wP3FiOcbbR4o0jYm67LphNfxJu/Do8cT2svzV86jrY9D1Vp5Kk9uypiDiBoncZwOeApyrMI0lqEK0qmkXE74FfAtdl5qL6RpIkqWWZuTAijgduALoCF2bmxIg4HZiQmWPLdftFxCTgDeCkslDWE/hr2ZvsFeCosmDWGObNhNu/C/edD93Whn2+Cbt9DrqtVXUySWrvPgucA3wdSOAWYEyliSRJDaG1Pc1+AnwKOCcifgdclJmP1y+WJEkty8xxwLillp1SM53Af5Wv2jbNFE/QXNH+12mbpG3kjQUw4SK47dswbxbsdDTs/TVYZ4Oqk0lSQyifonxY1TkkSY2nVUWzzLwZuDki+lLc8nJzREwBfgH8thw3RpIktaUnb4Yb/h9MfxyGvRdGfQc2emfVqSSpoUTEFhS3Zm6YmduVt+gflJlnVBxNktTOtfrxWhExADgGOBb4O/BjiifO3FSXZJIkdVbTHofffgwuORjeeB0OuxSOvtaCmSStml8AXwUWAGTmQ9jzTJLUCq0d0+wPwJbAb4ADM/P5ctUVETGhXuEkSepUXnsZbvsOjP8lrLUO7HcG7DIGuvWoOpkkNbJemXlfOablYo0zpqUkqTKtHdPsnMy8taUVmTmyDfNIktT5vLEAxl9QFMzmz4GdPgV7/z/oPbDqZJLUEUyPiHdQPASAiPgY8PzyN5EkqfVFs20i4u+ZOQsgIvoDh2fmT+oXTZKkDi4TnrgBbvw6zHgSNtsb9v82bLjC5xVIklrv88D5wFYRMRX4F3BktZEkSY2gtUWz4zLzvMUzmTkzIo6jeKqmJElaWS9OKgb5f+pWGLA5HHEljNgPlrx9SJK0GiKiK/C5zNwnInoDXTJzTtW5woYHvgAAIABJREFUJEmNobVFs64REZm5uEtzV2Ct+sWSJKmDmjcTbvkW3H8R9OgDo86CnY+Frt2rTiZJHU5mvhERe5bTr1adR5LUWFpbNLueYtD/n5fz/1EukyRJK+vRa2Hn42Cvk6HXelWnkaSO7u8RMRb4HfBm4Swzf19dJElSI2ht0ewrFIWy/yznbwIuqEsiSZI6srX7wwl/hx7rVJ1EkjqLnsAM4P01yxKwaCZJWq5WFc0ycxHw0/IlSZJWhwUzSaq7iDgcuDEzP1V1FklSY2pV0SwiRgDfAbah+KUGgMzcrE65JEmSJGl1DAV+FxHdgVuA64D7Fo/TLEnSinRpZbuLKHqZLQT2Bn4N/LZeoSRJHV9EnBgR60bhlxHxQETsV3UuSVLHkJlnZ+b7gQ8B/wA+DTwQEZdGxCcjYsNqE0qS2rvWFs3WzsxbgMjMZzLzNOCA+sWSJHUCn87MV4D9gP7AJ4Czqo0kSepoMnNOZv4hM/8jM3cEzgDWp+gIIEnSMrX2QQDzI6IL8GREHA9MBRyQRZK0OqL8+yHgN5k5MSJieRtIkrQqImIQsClvff8Zn5nfrzCSJKkBtLan2YlAL+AEYCfgKODoeoWSJHUK90fEjRRFsxsiog+wqOJMkqQOJiLOBv4GfB04qXx9uQ32OyoiHo+IyRFxcgvre0TEFeX6eyNiWM26r5bLH4+I/Vc3iySpPlbY0ywiugKHZuaXgbmAT5+RJLWFzwA7AE9l5msRsR6eYyRJbe8jwJaZOb+tdlh+RzoP2BdoAsZHxNjMnFTT7DPAzMzcPCIOA84GDo2IbYDDgG2BTYCbI2KLzHyjrfJJktrGCotmmflGROy5JsJIkjqV3YEHM/PViDgKeDfw44ozSZI6nqeA7kCbFc2AXYDJmfkUQERcDowGaotmo4HTyumrgHPLYQhGA5eXRbx/RcTkcn93t2G+N115xM6s89Kr9di1JLUbczfozSGXjm/z/bZ2TLO/R8RY4HfAm5+4mfn7Nk8kSeosfgq8KyLeBfw3cAHFoMz/VmkqSVJH8xrwYETcQk3hLDNPWI19DgKm1Mw3Absuq01mLoyI2cCAcvk9S207qKWDRMQYYAzA0KFDVyOuJGlVtLZo1hOYAby/ZlkCFs0kSatqYWZmRIwGzs3MX0bEZ6oOJUnqcMaWr4aTmecD5wOMHDkyV2Uf9eh5IUmdRauKZpnpGDOSpLY2JyK+CnwCeG/5lObuFWeSJHUwmXlxRKwFbFEuejwzF6zmbqcCQ2rmB5fLWmrTFBHdgL4UHRFas60kqR1oVdEsIi6i6Fm2hMz8dJsnkiR1FocCRwCfzswXImIo8L8VZ5IkdTARsRdwMfA0EMCQiDg6M+9Yjd2OB0ZExHCKgtdhFOe0WmOBoynGKvsY8Jeyh/VY4NKI+AHFgwBGAPetRhZJUp209vbMP9VM9wT+HXiu7eNIkjqLslB2CbBzRHwYuC8zf111LklSh/N9YL/MfBwgIrYALgN2WtUdlmOUHQ/cAHQFLszMiRFxOjAhM8cCvwR+Uw70/zJFYY2y3ZUUDw1YCHzeJ2dKUvvU2tszr66dj4jLgDvrkkiS1ClExCEUPctuo/jl//8i4qTMvKrSYJKkjqb74oIZQGY+ERGrPRxAZo4Dxi217JSa6Wbg48vY9kzgzNXNIEmqr9b2NFvaCGCDtgwiSep0vgbsnJkvAUTE+sDNgEUzSVJbmhARFwC/LeePBCZUmEeS1CBaO6bZHJYc0+wF4Ct1SSRJ6iy6LC6YlWYAXaoKI0nqsP4T+DxwQjn/V+An1cWRJDWK1t6e2afeQSRJnc71EXEDxbgyUDwYYNxy2kuStNIycz7wg/IlSVKrtban2b9TPO1ldjnfD9grM6+pZzhJUseVmSdFxMHAHuWi8zPzD1VmkiR1HBFxZWYeEhEPs+RdMwBk5vYVxJIkNZDWjml2au0XmcycFRGnAhbNJEmrrHzQzNUrbChJ0so7sfz74UpTSJIaVmvHjmmp3QoLbhExKiIej4jJEXFyC+uPiYhpEfFg+Tq2XL5DRNwdERMj4qGIOLSVOSVJ7VxEzImIV1p4zYmIV6rOJ0nqGDLz+XJyOjAlM58BegDvAp6rLJgkqWG0tqfZhIj4AXBeOf954P7lbRARXcv2+wJNwPiIGJuZk5ZqekVmHr/UsteAT2bmkxGxCXB/RNyQmbNamVeS1E45TqYkaQ27A3hvRPQHbgTGU4yjeWSlqSRJ7V5re5p9AXgduAK4HGimKJwtzy7A5Mx8KjNfL7cb3ZqDZeYTmflkOf0c8BKwfiuzSpIkSdJikZmvAR8FfpKZHwe2rTiTJKkBtPbpma8Cb7u9cgUGAVNq5puAXVtod3BEvA94AvhSZtZuQ0TsAqwF/HPpDSNiDDAGYOjQoSsZT5IkSVInEBGxO0XPss+Uy7pWmEeS1CBa1dMsIm4qn5i5eL5/RNzQBse/FhhWPrnmJuDipY67MfAb4FOZuWjpjTPz/MwcmZkj11/fjmiSJEmS3uaLwFeBP2TmxIjYDLi14kySpAbQ2jHNBtaOJ5aZMyNigxVsMxUYUjM/uFz2psycUTN7AfDdxTMRsS7wZ+BrmXlPK3NKkiRJ0psy83bg9pr5p4ATqkskSWoUrS2aLYqIoZn5LEBEDANyBduMB0ZExHCKYtlhwBG1DSJi45qn2hwEPFouXwv4A/DrzLyqlRklSZIkCYCI+FFmfjEirqWF7y6ZeVAFsSRJDaS1RbOvAXdGxO1AAO+lHEtsWTJzYUQcD9xAMWbAhWV36NOBCZk5FjghIg4CFgIvA8eUmx8CvA8YEBGLlx2TmQ+2+p1JkiRJ6sx+U/79XqUpJEkNq7UPArg+IkZSFMr+DlwDzGvFduOAcUstO6Vm+qsU4wssvd1vgd+2JpskSZIkLS0z7y8nJwDzFo+RHBFdgR6VBZMkNYxWFc0i4ljgRIpxyR4EdgPuBt5fv2iSJEmStNpuAfYB5pbzawM3Au+pLJEkqSG06umZFAWznYFnMnNvYEdg1vI3kSRJkqTK9czMxQUzyuleFeaRJDWI1hbNmjOzGSAiemTmY8CW9YslSZIkSW3i1Yh49+KZiNiJVgw1I0lSax8E0BQR/SjGMrspImYCz9QvliRJkiS1iS8Cv4uI5ygearYRcGi1kSRJjaC1DwL493LytIi4FegLXF+3VJIkSZLUBjJzfERsxVt3yjyemQuqzCRJagyt7Wn2psy8vR5BJEmSJKmtRUQv4L+ATTPzuIgYERFbZuafqs4mSWrfWjummSRJkiQ1oouA14Hdy/mpwBnVxZEkNQqLZpIkSZI6sndk5neBBQCZ+RrF2GaSJC2XRTNJUkOJiFER8XhETI6Ik5fR5pCImBQREyPi0prlZ0fEI+Xr0Jrll5T7fCQiLoyI7vV8D8/PnsfrCxfV8xCSpLe8HhFrAwkQEe8A5lcbSZLUCCyaSZIaRkR0Bc4DPghsAxweEdss1WYE8FVgj8zcluKpaUTEAcC7gR2AXYEvR8S65WaXAFsB7wTWBo6t13t4dsZr7P2927j4rqfrdQhJ0pJOpXiI2ZCIuAS4BfifaiNJkhqBRTNJUiPZBZicmU9l5uvA5cDopdocB5yXmTMBMvOlcvk2wB2ZuTAzXwUeAkaVbcZlCbgPGFyvNzB0QC9232wAP77lSV6a01yvw0iSgIjoAvQHPgocA1wGjMzM2yqMJUlqEBbNJEmNZBAwpWa+qVxWawtgi4j4W0TcExGjyuX/AEZFRK+IGAjsDQyp3bC8LfMTFD0S3iYixkTEhIiYMG3atFV+E6ccuC3zF77B2dc9vsr7kCStWGYuAv4nM2dk5p8z80+ZOb3qXJKkxmDRTJLU0XQDRgB7AYcDv4iIfpl5IzAOuIuip8HdwBtLbfsTit5of21px5l5fmaOzMyR66+//ioHHD6wN5/ZczOufqCJB56ducr7kSS1ys0R8eWIGBIR6y1+VR1KktT+WTSTJDWSqSzZO2xwuaxWEzA2Mxdk5r+AJyiKaGTmmZm5Q2buS/HktCcWbxQRpwLrA/9Vx/xvOv79m7NBnx6cNnYiixblmjikJHVWhwKfA24HJtS8JElaLotmkqRGMh4YERHDI2It4DBg7FJtrqHoZUZ5G+YWwFMR0TUiBpTLtwe2B24s548F9gcOL2/lqbt1enTjqx/aioeaZnPV/U1r4pCS1FltQ/EQmX8ADwL/B2xbaSJJUkOwaCZJahiZuRA4HrgBeBS4MjMnRsTpEXFQ2ewGYEZETAJuBU7KzBlAd+Cv5fLzgaPK/QH8DNgQuDsiHoyIU9bE+/nIDoPYadP+nH39Y8yet2BNHFKSOqOLga2BcygKZtuUyyRJWq5uVQeQJGllZOY4irHJapedUjOdFLdY/tdSbZopvii1tM9KzocRwTcP2pYDz72TH9/8JKcc2GI8SdLq2S4zaz9gby1/QJEkabnsaSZJUoW2G9SXw3YeysV3P82TL86pOo4kdUQPRMRui2ciYlcc00yS1AoWzSRJqtiX99uC3mt15bRrJ1J0lJMktaGdgLsi4umIeJri6ck7R8TDEfFQtdEkSe2Zt2dKklSxAev04L/325JTx07khokvMmq7jaqOJEkdyaiqA0iSGpNFM0mS2oEjdx3Kpfc+yxl/nsReW65Pz+5dq44kSR1CZj5TdQZJUmPy9kxJktqBbl27cOpB29A0cx4/v/2pquNIkiRJnZ5FM0mS2on3vGMgB7xzY35y22SaZr5WdRxJUgsiYr2IuCkiniz/9l9Gu6PLNk9GxNHlsl4R8eeIeCwiJkbEWWs2vSRpZVg0kySpHfl/B2xNBHxn3GNVR5Ektexk4JbMHAHcUs4vISLWA04FdgV2AU6tKa59LzO3AnYE9oiID66Z2JKklWXRTJKkdmRQv7X53F6b8+eHn+euydOrjiNJervRwMXl9MXAR1posz9wU2a+nJkzgZuAUZn5WmbeCpCZrwMPAIPXQGZJ0iqwaCZJUjsz5n2bMbj/2px27UQWvrGo6jiSpCVtmJnPl9MvABu20GYQMKVmvqlc9qaI6AccSNFbrUURMSYiJkTEhGnTpq1eaknSSrNoJklSO9Oze1e+fsA2PPHiXH57jw99k6Q1LSJujohHWniNrm2XmQnkKuy/G3AZcE5mLvPpL5l5fmaOzMyR66+//kq/D0nS6ulWdQBJkvR2+2+7Ie8dMZAf3PQEB75rEwas06PqSJLUaWTmPstaFxEvRsTGmfl8RGwMvNRCs6nAXjXzg4HbaubPB57MzB+1QVxJUp3Y00ySpHYoIjj1wG147fU3+N6NT1QdR5L0lrHA0eX00cAfW2hzA7BfRPQvHwCwX7mMiDgD6At8cQ1klSStBotmkiS1U5tv0Iej3zOMy8c/y8NNs6uOI0kqnAXsGxFPAvuU80TEyIi4ACAzXwa+BYwvX6dn5ssRMRj4GrAN8EBEPBgRx1bxJiRJK+btmZIktWMn7jOCPz44lVPHPsLV//keIqLqSJLUqWXmDOADLSyfABxbM38hcOFSbZoAP8glqUHY00ySpHZs3Z7d+Z9RW/HAs7O45sGpVceRJEmSOg2LZpIktXMfe/dg3jW4L98Z9xhz5y+sOo4kSZLUKVg0kySpnevSJTjtoG15ac58/u8vT1YdR5IkSeoULJpJktQAdhzan4/tNJgL7/wXT02bW3UcSZIkqcOzaCZJUoP4yqit6NmtK9/606Sqo0iSJEkdnk/PlCSpQazfpwcn7jOCM/78KH957EXev9WGVUeSJEnq9BYsWEBTUxPNzc1VR9Fy9OzZk8GDB9O9e/dWb2PRTJKkBvLJ3Ydx2X3P8o1rJrLt5/qy4bo9q44kSZLUqTU1NdGnTx+GDRtGRFQdRy3ITGbMmEFTUxPDhw9v9XbenilJUgNZq1sXvn/IDsx67XUO/8U9vPSKv2hKkiRVqbm5mQEDBlgwa8ciggEDBqx0b0CLZpIkNZgdhvTjV5/ehRdmN3PEBfcybc78qiNJkiR1ahbM2r9V+T+qa9EsIkZFxOMRMTkiTm5h/TERMS0iHixfx9asuz4iZkXEn+qZUZKkRrTzsPW46JidmTpzHkdecA/T51o4kyRJktpS3YpmEdEVOA/4ILANcHhEbNNC0ysyc4fydUHN8v8FPlGvfJIkNbpdNxvAhcfszLMvv8ZRF9zLy6++XnUkSZIkrWGzZs3iJz/5ySpt+6EPfYhZs2a1caKVN2zYMKZPnw7AOuusU3Gat9Szp9kuwOTMfCozXwcuB0a3duPMvAWYU69wkiR1BLu/YwC/PHpn/jX9VY684F5mWjiTJEnqVJZXNFu4cOFytx03bhz9+vVbpeOuaN8dQT2fnjkImFIz3wTs2kK7gyPifcATwJcyc0oLbVoUEWOAMQBDhw5djaiSJDWuPTYfyC8+OZJjfz2Bo355L5ccuyv9eq1VdSxJkqRO55vXTmTSc6+06T632WRdTj1w22WuP/nkk/nnP//JDjvswL777ssBBxzAN77xDfr3789jjz3GE088wUc+8hGmTJlCc3MzJ554ImPGjAGKHl4TJkxg7ty5/P/27jw+qvre//jrk4WEsCYBIQaUsC+CIAFBUBGqQrGiKESrFlyLLEq9tqX2p7WW3qql3kovglURsVwFUa7opcoiiBURQgiyE6AghH0nQCDL9/fHnMQQkrBlZjLk/Xw85jHnfM/3nPOZM5P5wGe+55w+ffrQvXt3Fi1aRGJiIh9//DFVq1Y9bV+DBw8mOjqa5cuX061bN4YNG8awYcPYu3cvMTExvPHGG7Rs2ZLdu3czZMgQNm/eDMD48eO57rrrSo2jovJn0excfAK855w7aWY/B94Bep7rys65vwN/B0hOTnb+CVFERKTiu6F5Xf7+QEcem7yMB95awj8evpZaMZHBDktERERE/OzFF19k1apVpKenA7BgwQLS0tJYtWoVSUlJAEycOJG4uDhOnDhBp06duOuuu4iPjz9tOxkZGbz33nu88cYbDBw4kA8//JD777//jP1t376dRYsWER4eTq9evZgwYQLNmjXj22+/ZejQoXzxxRc88cQT3HjjjcyYMYO8vDyysrLOOY6KxJ9Fs0ygYZH5Bl5bIefc/iKzbwIv+zEeERGRS1qPFpcx4YFr+Pm7y/jZxG9595FrqRmtwpmIiIhIoJQ1IiyQOnfuXFgwAxg7diwzZswAYNu2bWRkZJxRrEpKSqJ9+/YAdOzYkS1btpS47QEDBhAeHk5WVhaLFi1iwIABhctOnvTdnOqLL75g8uTJAISHh1OrVq1zjqMi8WfRbCnQzMyS8BXL7gF+WrSDmSU453Z6s7cDa/0Yj4iIyCWvZ8t6jL+vI49PWcagiUuY/FBnaqhwJiIiIlKpVKtWrXB6wYIFzJ07l2+++YaYmBh69OhBdnb2GetERUUVToeHh3PixIkyt52fn0/t2rULR7idzbnGUZH47UYAzrlcYDjwOb5i2DTn3Goze8HMbve6PWFmq81sBfAEMLhgfTP7CvgA6GVm283sVn/FKiIicin5Uet6/PdPr2Hl9sMMfnspWScv/Yu0ioiIiFRWNWrU4OjR0u+jePjwYWJjY4mJiWHdunUsXry4XPZbs2ZNkpKS+OCDDwBwzrFixQoAevXqxfjx4wHIy8vj8OHDfovDn/x590ycc7Occ82dc02cc3/02p5zzs30pn/jnGvjnLvaOXeTc25dkXWvd87Vdc5Vdc41cM597s9YRURELiW3tqnP3+7tQPq2Qzz49hKOqXAmIiIickmKj4+nW7duXHXVVfzyl788Y3nv3r3Jzc2lVatWjBo1ii5dupTbvqdMmcJbb73F1VdfTZs2bfj4448BePXVV5k/fz5t27alY8eOrFmzxq9x+Is5d2lcPz85OdmlpqYGOwwRkUuSmS1zziUHO46KJFTyzv99t5Mn3l9OxytjmfRgJ2KqBPseQCIiZ6e8c6ZQyTsildHatWtp1apVsMOQc1DSe1VWzvHrSDMREREJrr7tEvivlPakbjnAw5NSOXEqL9ghiYiIiIiEBBXNRERELnG3X305rwxsz7f/3s+jk1PJzlHhTERERETkbFQ0ExERqQTu6JDIn+++mq837VPhTERERETkHKhoJiIiUknc1bEBL93Vjn9t3MfP312mwpmIiIiISBlUNBMREalEBiY35MX+bflyw17ufG0RG3aXfntyEREREZHKTEUzERGRSial0xW8NSiZPUey+cnf/sWkr//NpXI3bRERERGR8qKimYiISCXUq1U9Pht5A92a1uH5T9Yw6O2l7DmSHeywRERERCQAqlevHrB9TZo0ieHDhwPw/PPPM2bMmIDt+2KpaCYiIlJJ1a0RxVuDkvnDHVex5N/7ufWvC/l89a5ghyUiIiIiQeacIz8/P9hhBF1EsAMQERGR4DEzHuhyJV0bxzNy6nJ+/u4y7unUkGdva021KP0zQUREROS8/HMU7FpZvtus3xb6vFjq4lGjRtGwYUOGDRsG+EZzVa9enSFDhtCvXz8OHjxITk4Oo0ePpl+/fqVuZ8uWLdx6661ce+21LFu2jFmzZjFt2jSmTZvGyZMnufPOO/n9738PwOTJkxkzZgxmRrt27Xj33Xf55JNPGD16NKdOnSI+Pp4pU6ZQr1698j0WAaaRZiIiIkLTy6rz0ePdGNqjCVNTt9F37Fcs//5gsMMSERERkbNISUlh2rRphfPTpk0jJSWF6OhoZsyYQVpaGvPnz+c//uM/znod24yMDIYOHcrq1atZv349GRkZLFmyhPT0dJYtW8bChQtZvXo1o0eP5osvvmDFihW8+uqrAHTv3p3FixezfPly7rnnHl5++WW/vu5A0E/IIiIiAkCViDB+1bslNzavy1PTVnD3hG94omczht3UhIhw/c4mIiIiclZljAjzlw4dOrBnzx527NjB3r17iY2NpWHDhuTk5PDMM8+wcOFCwsLCyMzMZPfu3dSvX7/UbV155ZV06dIFgNmzZzN79mw6dOgAQFZWFhkZGaxYsYIBAwZQp04dAOLi4gDYvn07KSkp7Ny5k1OnTpGUlOTnV+5/+hewiIiInObaxvHMevJ6ftIugf+au4GBr3/D1v3Hgh2WiIiIiJRiwIABTJ8+nalTp5KSkgLAlClT2Lt3L8uWLSM9PZ169eqRnV32jZ+qVatWOO2c4ze/+Q3p6emkp6ezceNGHn744VLXHTFiBMOHD2flypW8/vrrZ91XKFDRTERERM5Qq2okf72nA6/e056MPVn8+NWvmJa67axD+kVEREQk8FJSUnj//feZPn06AwYMAODw4cNcdtllREZGMn/+fLZu3Xpe27z11luZOHEiWVlZAGRmZrJnzx569uzJBx98wP79+wE4cOBA4f4SExMBeOedd8rrpQWVimYiIhJSzKy3ma03s41mNqqUPgPNbI2ZrTaz/ynS/pKZrfIeKUXak8zsW2+bU82sSiBeSyjo1z6Rz0bewFWJtfjV9O8YOiWNg8dOBTssERERESmiTZs2HD16lMTERBISEgC47777SE1NpW3btkyePJmWLVue1zZvueUWfvrTn9K1a1fatm3L3XffzdGjR2nTpg2//e1vufHGG7n66qt56qmnAN8NCAYMGEDHjh0LT90MdXap/GKcnJzsUlNTgx2GiMglycyWOeeSK0Ac4cAG4GZgO7AUuNc5t6ZIn2bANKCnc+6gmV3mnNtjZn2BkUAfIApYAPRyzh0xs2nAR865981sArDCOTe+rFgqW97Jy3e88dVm/jJ7PXHVqjBmwNVc36xusMMSkUtURck7FUllyzsioWTt2rW0atUq2GHIOSjpvSor52ikmYiIhJLOwEbn3Gbn3CngfaD4fbMfBcY55w4COOf2eO2tgYXOuVzn3DHgO6C3mRnQE5ju9XsHuMPPryPkhIcZQ25swoyh3ageFcEDby3huY9XsS/rZLBDExERERHxCxXNREQklCQC24rMb/faimoONDezr81ssZn19tpX4CuSxZhZHeAmoCEQDxxyzuWWsU0AzOwxM0s1s9S9e/eW00sKLVcl1uLTEdcz+LpG/GPxVm54eT4vfbZOp2yKiIiIyCVHRTMREbnURADNgB7AvcAbZlbbOTcbmAUsAt4DvgHyzmfDzrm/O+eSnXPJdetW3lMTq1YJ5/nb2zDnqRv5Uat6TPhyE9e/PJ9XZq/n8ImcYIcnIuJXZhZnZnPMLMN7ji2l3yCvT4aZDSph+UwzW+X/iEVE5EKpaCYiIqEkE9/osAINvLaitgMznXM5zrl/47sGWjMA59wfnXPtnXM3A+Yt2w/UNrOIMrYpJWhStzpj7+3A5yNv4IbmdRj7xUa6v/QFY+dlcDRbxTMRuWSNAuY555oB87z505hZHPA74Fp8lxb4XdHimpn1B7ICE66IiFwoFc1ERCSULAWaeXe7rALcA8ws1ud/8Y0ywzsNszmw2czCzSzea28HtANmO98dceYDd3vrDwI+9vcLuZQ0r1eD1+7ryKwnrqdL43hembOB61+ez2sLNnLsZO7ZNyAiElr64bv+JZR+HcxbgTnOuQPeNTbnAL0BzKw68BQwOgCxiojIRVDRTEREQoZ33bHhwOfAWmCac261mb1gZrd73T4H9pvZGnzFsF865/YDkcBXXvvfgfuLXMfs18BTZrYR3zXO3grcq7p0tL68Jm/8LJlPhnenQ8PavPzZem54eT5vLNzMiVPndSasiEhFVs85t9Ob3gXUK6FPWdfg/APwF+D42Xaka2mKiASXimYiIhJSnHOznHPNnXNNnHN/9Nqec87N9Kadc+4p51xr51xb59z7Xnu219baOdfFOZdeZJubnXOdnXNNnXMDnHO6JeRFaNugFm8/2JmPhl5H68tr8sdZa7nhz/N5++t/k52j4pmIVHxmNtfMVpXwOO2Ozd5oZXce220PNHHOzTiX/rqWpoici0OHDvHNcht8AAAb10lEQVTaa69d0Lo//vGPOXToUDlHdP4eeeQR1qxZU+ry5557jrlz5wYwIp+Is3cREREROX/XXBHLuw9fy5J/H+CVOev5/SdreP3LzQzr2ZSU5IZUidBvdyJSMTnnflTaMjPbbWYJzrmdZpYA7CmhWybepQI8DYAFQFcg2cy24Pu/2GVmtsA51wMRkQtUUDQbOnToGctyc3OJiCi99DNr1qxyj+ds+yzJm2++WebyF1544WJCumAqmomIiIhfdU6K4/3HurJo4z7+MmcDz/7vKiYs2MSInk2585pEoiLCgx2iiMj5mInv+pcvUvp1MD8H/rPIxf9vAX7jnDsAjAcws0bApyqYiVxaXlryEusOrCvXbbaMa8mvO/+61OWjRo1i06ZNtG/fnptvvpm+ffvy7LPPEhsby7p169iwYQN33HEH27ZtIzs7myeffJLHHnsMgEaNGpGamkpWVhZ9+vShe/fuLFq0iMTERD7++GOqVq162r4GDx5MdHQ0qampHDlyhFdeeYXbbruNSZMm8dFHH5GVlUVeXh6zZs1ixIgRrFq1ipycHJ5//nn69etHXl4ev/71r/nss88ICwvj0UcfZcSIEfTo0YMxY8bQoUMHHn74YVJTUzEzHnroIX7xi18wePBgbrvtNu6++27mzZvH008/TW5uLp06dWL8+PFERUXRqFEjBg0axCeffEJOTg4ffPABLVu2vKhjr6KZiIiIBMR1TevQtUk8CzP28cqcDYz6aCV/+uc6fnJ1Av2vaUCHhrUxs2CHKSJyNi8C08zsYWArMBDAzJKBIc65R5xzB8zsD/huYAPwglcwExEpdy+++CKrVq0iPd139ZEFCxaQlpbGqlWrSEpKAmDixInExcVx4sQJOnXqxF133UV8fPxp28nIyOC9997jjTfeYODAgXz44Yfcf//9Z+xvy5YtLFmyhE2bNnHTTTexceNGANLS0vjuu++Ii4vjmWeeoWfPnkycOJFDhw7RuXNnfvSjHzF58mS2bNlCeno6ERERHDhw+ldjeno6mZmZrFq1CuCMU0ezs7MZPHgw8+bNo3nz5vzsZz9j/PjxjBw5EoA6deqQlpbGa6+9xpgxY846gu1sVDQTERGRgDEzbmxelxua1eHrjfuZvmwb05dt5x+Lv6dxnWr0vyaROzok0iA2JtihioiUyLu5TK8S2lOBR4rMTwQmlrGdLcBVfghRRIKorBFhgdS5c+fCghnA2LFjmTHDdznFbdu2kZGRcUbRLCkpifbt2wPQsWNHtmzZUuK2Bw4cSFhYGM2aNaNx48asW+cbWXfzzTcTFxcHwOzZs5k5cyZjxowBfMWu77//nrlz5zJkyJDC0zcL+hdo3LgxmzdvZsSIEfTt25dbbrnltOXr168nKSmJ5s2bAzBo0CDGjRtXWDTr379/YfwfffTReRyxkqloJiIiIgFnZnRvVofuzepwNDuHf67axYfLtjNm9gbGzN5A18bx9L8mkT5tE6gepX+uiIiIiJyPatWqFU4vWLCAuXPn8s033xATE0OPHj3Izs4+Y52oqKjC6fDwcE6cOFHitoufGVAwX3Sfzjk+/PBDWrRocV5xx8bGsmLFCj7//HMmTJjAtGnTmDix1N8fzlDwGsLDw8nNzT2vfZdEV+AVERGRoKoRHcnA5IZM/XlXvvrVTTx1c3N2Hj7BL6d/R6fRc/nF1HS+ythLXv4536BOREREpNKoUaMGR48eLXX54cOHiY2NJSYmhnXr1rF48eKL2t8HH3xAfn4+mzZtYvPmzSUWxm699Vb+9re/4bvJMCxfvhzwjUZ7/fXXCwtaxU/P3LdvH/n5+dx1112MHj2atLS005a3aNGCLVu2FJ4S+u6773LjjTde1Ospi366FRERkQqjYVwMT/RqxoieTUn7/iAfpmXyyYodzFieSf2a0dzRIZG7OybS9LIawQ5VREREpEKIj4+nW7duXHXVVfTp04e+ffuetrx3795MmDCBVq1a0aJFC7p06XJR+7viiivo3LkzR44cYcKECURHR5/R59lnn2XkyJG0a9eO/Px8kpKS+PTTT3nkkUfYsGED7dq1IzIykkcffZThw4cXrpeZmcmDDz5Ifn4+AH/6059O2250dDRvv/02AwYMKLwRwJAhQy7q9ZTFCqp+oS45OdmlpqYGOwwRkUuSmS1zziUHO46KRHkncLJz8pi3dg8fpm3nyw2+EWftGtSif4dEfnL15cRXjzr7RkQkpCjvnEl5R6TiWrt2La1atQp2GAFR9C6Woaik96qsnKORZiIiIlKhRUeG07ddAn3bJbD36Ek+Ts/ko7RMnv9kDc9/sobWCTXp1jSe65rWoXOjOKrpGmgiIiIiUg70r0oREREJGXVrRPHI9Y155PrGrN15hLlrdvP1pn28s2grb3z1byLDjfYNa3NdE99NBq5uUJsqEbqEq4iIiEh5mDRpUrBDCCgVzURERCQktUqoSauEmozo1YwTp/JI3XqArzfuZ9GmfYz9IoNX52UQUyWczklxdGtSh+uaxtOqfk3CwuzsGxcRERGRSk9FMxEREQl5VauEc32zulzfrC4Ah46fYvHm/Xy9cT9fb9rHgvVrAYirVoWujeO5rmk83ZrU4cr4mDNumy4iIiIiAiqaiYiIyCWodkwVel+VQO+rEgDYefgEi7wC2qKN+/m/lTsBSKxdlXYNatGyfk1aJtSgVf2aNIitqtFoIiIiIqKimYiIiFz6EmpV5a6ODbirYwOcc2zae4xFm/axePN+1uw4wmerd1FwQ/HqURG0qF+DlvVr0DKhJq3q16BF/RrUiI4M7osQERERkYBS0UxEREQqFTOj6WXVaXpZdX7WtREAx07msmH3UdbtOsranUdYt/MoM1fsYMq33xeu1zCuKi3r+4poLRNq0rJ+Da6Mr0a4RqWJiIhIiKlevTpZWVkB2VdqaiqTJ09m7NixJS7fsWMHTzzxBNOnTw9IPOdDRTMRERGp9KpFRdDhilg6XBFb2OacY8fhbNbtPPJDMW3XUeat3U2+NyqtamQ4jetWI7F2VS6vXZUGsb7ny2tXJbF2VepUr6JrpomIiMglJS8vj/Dw8HPun5ycTHJycqnLL7/88gpZMAMVzURERERKZGYkesWvXq3qFbZn5+SRsTuLtbt8I9I278tiy/5jfL1xH8dO5Z22jSoRYV5BLZrLa1Ul0SuqNfAKawm1o4mKOPd/dIqIiEjFtus//5OTa9eV6zajWrWk/jPPlLp81KhRNGzYkGHDhgHw/PPPU716dYYMGUK/fv04ePAgOTk5jB49mn79+pW6nS1bttC7d286duxIWloabdq0YfLkycTExNCoUSNSUlKYM2cOv/rVr4iLi+N3v/sdJ0+epEmTJrz99ttUr16dpUuX8uSTT3Ls2DGioqKYN28ey5YtY8yYMXz66ad8+eWXPPnkk4Dv31oLFy5k//793HbbbaxatYrs7Gwef/xxUlNTiYiI4JVXXuGmm25i0qRJzJw5k+PHj7Np0ybuvPNOXn755XI9ziVR0UxERETkPERHhtO2QS3aNqh1WrtzjiMncsk8dILMQyfY4T0XTH+5YS97jp48Y3t1qkcRX60KtWMiiY2pQmy1KsR60z+0RVI7pgqxMVWoVTVSp4SKiIhIoZSUFEaOHFlYNJs2bRqff/450dHRzJgxg5o1a7Jv3z66dOnC7bffXuYo+PXr1/PWW2/RrVs3HnroIV577TWefvppAOLj40lLS2Pfvn3079+fuXPnUq1aNV566SVeeeUVRo0aRUpKClOnTqVTp04cOXKEqlWrnrb9MWPGMG7cOLp160ZWVhbR0dGnLR83bhxmxsqVK1m3bh233HILGzZsACA9PZ3ly5cTFRVFixYtGDFiBA0bNizPQ3kGvxbNzKw38CoQDrzpnHux2PLBwJ+BTK/pv51zb3rLBgH/z2sf7Zx7x5+xioiIiFwMM6NWTCS1YiJpfXnNEvuczM1j1+FsXzHt4Al2HMpmx6ETHDh+ikPHT7FxbxaHtp7i4PEc8grOAT1jP1AzOpLYmIJCWiQ1q0YSUyWcqpERVIsKp2qVcGIiw4mpEuGbruKbjvGmqxaZj4oI0ymkIiIi5aSsEWH+0qFDB/bs2cOOHTvYu3cvsbGxNGzYkJycHJ555hkWLlxIWFgYmZmZ7N69m/r165e6rYYNG9KtWzcA7r//fsaOHVtYNEtJSQFg8eLFrFmzprDfqVOn6Nq1K+vXrychIYFOnToBULPmmf8e6tatG0899RT33Xcf/fv3p0GDBqct/9e//sWIESMAaNmyJVdeeWVh0axXr17UquX70bJ169Zs3bo1dItmZhYOjANuBrYDS81spnNuTbGuU51zw4utGwf8DkgGHLDMW/egv+IVERER8beoiHCujK/GlfHVyuznnOPoyVwOHcvh4PFTHDx+ikPHC6ZzOFTkeW/WSTbtPcbxU3mcOJXL8Zy8wjuBnoswg5gqEVSJCCMy3IgMD6NKeBiR4WFERljhtG/56X1+aAsjItwIMyM8DMLDwgj3psPCzJsuWG6FbREF02EQZr7lZr5pw1eILDofFgaGr83MCDPffJgBVmSZ99oKioGnt0HBXEGt0Ir2LXZ8itcTrVgPf9Qby2ObxeMMtJgq4TSqU/bnXEREys+AAQOYPn06u3btKixuTZkyhb1797Js2TIiIyNp1KgR2dnZZW6n+A9pReerVfN9rzvnuPnmm3nvvfdO67ty5cqzxjlq1Cj69u3LrFmz6NatW+GIuHMRFRVVOB0eHk5ubu45rXcx/DnSrDOw0Tm3GcDM3gf6AcWLZiW5FZjjnDvgrTsH6A28V+ZaF+qfo2DX2d9cEZGQVb8t9Hnx7P1EpEIwM2pGR1IzOpIr4mPOa13nHCdz8zl2MtdXSMvJ4/ipPI6fyuXEqR+mj3vTBW05efnk5OVzKi+fnDxHTu4P86e87eXkuSJ98snJ/WE+N8+R5xx5+a7UUXJSeXRuFMe0IV2DHYaISKWRkpLCo48+yr59+/jyyy8BOHz4MJdddhmRkZHMnz+frVu3nnU733//Pd988w1du3blf/7nf+jevfsZfbp06cKwYcPYuHEjTZs25dixY2RmZtKiRQt27tzJ0qVL6dSpE0ePHj3j9MxNmzbRtm1b2rZty9KlS1m3bh3t27cvXH799dczZcoUevbsyYYNG/j+++9p0aIFaWlpF3mELow/i2aJwLYi89uBa0vod5eZ3QBsAH7hnNtWyrqJxVc0s8eAxwCuuOKKcgpbREREJHSZGdGR4URHhhMfxDjy838oouUXKablee35+XjPjtzCQpsj34FzkO9c4TOcPu/wFQd9877pfAcO5ztHgcInXEF74XRBu/uhX+E6pxf7io/YO2O+hNftzmeYX0nrX9TaBTFc9BY4c8zd+YmNibzYIERE5Dy0adOGo0ePkpiYSEJCAgD33XcfP/nJT2jbti3Jycm0bNnyrNtp0aIF48aN46GHHqJ169Y8/vjjZ/SpW7cukyZN4t577+XkSd/1WkePHk3z5s2ZOnUqI0aM4MSJE1StWpW5c+eetu5f//pX5s+fT1hYGG3atKFPnz7s3LmzcPnQoUN5/PHHadu2LREREUyaNOm0EWaBZheb2EvdsNndQG/n3CPe/APAtUVPxTSzeCDLOXfSzH4OpDjneprZ00C0c2601+9Z4IRzbkxp+0tOTnapqal+eS0iIpWdmS1zzpV+n+hKSHlHRMR/lHfOpLwjUnGtXbuWVq1aBTuMi7Zly5bCu1heqkp6r8rKOWF+jCUTKHpFtgb8cMF/AJxz+51zBbeRehPoeK7rioiIiIiIiIiI+Is/i2ZLgWZmlmRmVYB7gJlFO5hZQpHZ24G13vTnwC1mFmtmscAtXpuIiIiIiIiIiJSzRo0aXdKjzC6E365p5pzLNbPh+Ipd4cBE59xqM3sBSHXOzQSeMLPbgVzgADDYW/eAmf0BX+EN4IWCmwKIiIiIiIiIiFQkzrkz7jwpFcuFXJ7MnzcCwDk3C5hVrO25ItO/AX5TyroTgYn+jE9ERERERERE5GJER0ezf/9+4uPjVTiroJxz7N+/n+jo6PNaz69FMxERERERERGRS1mDBg3Yvn07e/fuDXYoUobo6GgaNGhwXuuoaCYiIiIiIiIicoEiIyNJSkoKdhjiB/68EYCIiIiIiIiIiEhIUtFMRERERERERESkGBXNREREREREREREirELueVmRWRme4GtF7GJOsC+cgrHX0IhRgiNOBVj+VCM5aeix3mlc65usIOoSJR3KgzFWH5CIU7FWD5CIUblnWIuMu+EwnseCjFCaMSpGMtHKMQIoRFnRY+x1JxzyRTNLpaZpTrnkoMdR1lCIUYIjTgVY/lQjOUnVOKU8hMK77liLB+hECOERpyKsXyEQoxSvkLhPQ+FGCE04lSM5SMUYoTQiDMUYiyNTs8UEREREREREREpRkUzERERERERERGRYlQ0+8Hfgx3AOQiFGCE04lSM5UMxlp9QiVPKTyi854qxfIRCjBAacSrG8hEKMUr5CoX3PBRihNCIUzGWj1CIEUIjzlCIsUS6ppmIiIiIiIiIiEgxGmkmIiIiIiIiIiJSjIpmIiIiIiIiIiIixVS6opmZ9Taz9Wa20cxGlbA8ysymesu/NbNGAY6voZnNN7M1ZrbazJ4soU8PMztsZune47lAxujFsMXMVnr7Ty1huZnZWO84fmdm1wQhxhZFjlG6mR0xs5HF+gT8WJrZRDPbY2arirTFmdkcM8vwnmNLWXeQ1yfDzAYFOMY/m9k67/2cYWa1S1m3zM+Gn2N83swyi7yfPy5l3TK/BwIQ59QiMW4xs/RS1g3IsRT/qeg5x4tBead84quQOcfbr/KO/2KsUHlHOUcqet4JlZzjxaG8c2FxKef4N07lnUBzzlWaBxAObAIaA1WAFUDrYn2GAhO86XuAqQGOMQG4xpuuAWwoIcYewKdBPpZbgDplLP8x8E/AgC7AtxXgvd8FXBnsYwncAFwDrCrS9jIwypseBbxUwnpxwGbvOdabjg1gjLcAEd70SyXFeC6fDT/H+Dzw9Dl8Fsr8HvB3nMWW/wV4LpjHUg+/vfcVPud4+1Xe8c97XyFyjrdf5R3/xVih8o5yTuV+hELeCZWc48WhvHNhsSjn+DdO5Z0APyrbSLPOwEbn3Gbn3CngfaBfsT79gHe86elALzOzQAXonNvpnEvzpo8Ca4HEQO2/HPUDJjufxUBtM0sIYjy9gE3Oua1BjAEA59xC4ECx5qKfu3eAO0pY9VZgjnPugHPuIDAH6B2oGJ1zs51zud7sYqCBP/Z9rko5jufiXL4Hyk1ZcXrfLQOB9/y1fwmqCp9zQHnHTypMzgHlnfISCnlHOafSq/B55xLKOaC8UyLlnPKjvFMxVLaiWSKwrcj8ds78ki7s4/3RHAbiAxJdMd5w6Q7AtyUs7mpmK8zsn2bWJqCB+ThgtpktM7PHSlh+Lsc6kO6h9D/WYB9LgHrOuZ3e9C6gXgl9KtIxfQjfL2slOdtnw9+Ge8OqJ5Yy9LsiHcfrgd3OuYxSlgf7WMrFCamcA8o75aii5xxQ3ilPoZJ3lHMufSGVdyp4zgHlnfKknFO+lHcCqLIVzUKGmVUHPgRGOueOFFuchm/o7dXA34D/DXR8QHfn3DVAH2CYmd0QhBjOiZlVAW4HPihhcUU4lqdxvrGqLthxlMbMfgvkAlNK6RLMz8Z4oAnQHtiJbzhwRXYvZf/yEjJ/ZxL6lHfKR6jlHFDeuUihlHeUc6TCCIGcAyHyNxFqeUc556Ip7wRYZSuaZQINi8w38NpK7GNmEUAtYH9AovOYWSS+JDLFOfdR8eXOuSPOuSxvehYQaWZ1Ahmjcy7Te94DzMA3BLSocznWgdIHSHPO7S6+oCIcS8/uguHc3vOeEvoE/Zia2WDgNuA+L+Gd4Rw+G37jnNvtnMtzzuUDb5Sy76AfRyj8fukPTC2tTzCPpZSLkMg53r6Vd8pPKOQcUN4pF6GSd5RzKo2QyDuhkHO8fSvvlB/lnHKivBN4la1othRoZmZJXkX+HmBmsT4zgYI7ddwNfFHaH4w/eOf9vgWsdc69Ukqf+gXXHjCzzvjex4AlOzOrZmY1CqbxXTRxVbFuM4GfmU8X4HCRIbmBVmqFO9jHsoiin7tBwMcl9PkcuMXMYr1huLd4bQFhZr2BXwG3O+eOl9LnXD4b/oyx6HUk7ixl3+fyPRAIPwLWOee2l7Qw2MdSykWFzzmgvOMHoZBzQHmnvGIMlbyjnFM5VPi8Ewo5x9uv8k75Us4pJ8o7QeAqwN0IAvnAd5eTDfjuJvFbr+0FfH8cANH4hrZuBJYAjQMcX3d8w1W/A9K9x4+BIcAQr89wYDW+u2AsBq4LcIyNvX2v8OIoOI5FYzRgnHecVwLJQXq/q+FLDLWKtAX1WOJLajuBHHznlz+M71oS84AMYC4Q5/VNBt4ssu5D3mdzI/BggGPciO/c+ILPZcGdly4HZpX12QhgjO96n7fv8CWGhOIxevNnfA8EMk6vfVLB57BI36AcSz38+v5X6JzjxaC8U35xVric4+1Xecd/MVaovFNSjF77JJRzKsWjpM8aFSjvEAI5x4tBeefCY1LO8W+cyjsBfpgXsIiIiIiIiIiIiHgq2+mZIiIiIiIiIiIiZ6WimYiIiIiIiIiISDEqmomIiIiIiIiIiBSjopmIiIiIiIiIiEgxKpqJiIiIiIiIiIgUo6KZSAVlZj3M7NNgxyEiIpWD8o6IiASKco6EChXNREREREREREREilHRTOQimdn9ZrbEzNLN7HUzCzezLDP7LzNbbWbzzKyu17e9mS02s+/MbIaZxXrtTc1srpmtMLM0M2vibb66mU03s3VmNsXMLGgvVEREKgTlHRERCRTlHKnsVDQTuQhm1gpIAbo559oDecB9QDUg1TnXBvgS+J23ymTg1865dsDKIu1TgHHOuauB64CdXnsHYCTQGmgMdPP7ixIRkQpLeUdERAJFOUcEIoIdgEiI6wV0BJZ6P4xUBfYA+cBUr88/gI/MrBZQ2zn3pdf+DvCBmdUAEp1zMwCcc9kA3vaWOOe2e/PpQCPgX/5/WSIiUkEp74iISKAo50ilp6KZyMUx4B3n3G9OazR7tlg/d4HbP1lkOg/9zYqIVHbKOyIiEijKOVLp6fRMkYszD7jbzC4DMLM4M7sS39/W3V6fnwL/cs4dBg6a2fVe+wPAl865o8B2M7vD20aUmcUE9FWIiEioUN4REZFAUc6RSk+VXJGL4JxbY2b/D5htZmFADjAMOAZ09pbtwXctAIBBwAQvUWwGHvTaHwBeN7MXvG0MCODLEBGREKG8IyIigaKcIwLm3IWOpBSR0phZlnOuerDjEBGRykF5R0REAkU5RyoTnZ4pIiIiIiIiIiJSjEaaiYiIiIiIiIiIFKORZiIiIiIiIiIiIsWoaCYiIiIiIiIiIlKMimYiIiIiIiIiIiLFqGgmIiIiIiIiIiJSjIpmIiIiIiIiIiIixfx/kOLC/89I8AoAAAAASUVORK5CYII=\n"
          },
          "metadata": {
            "needs_background": "light"
          }
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Prediction"
      ],
      "metadata": {
        "id": "nvbDGF6jDhn5"
      }
    },
    {
      "cell_type": "code",
      "source": [
        "# prediction on test_pts\n",
        "pts_model.predict(x_test_pts), y_test_pts"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "E0DxNwiqYm7s",
        "outputId": "fb7a5b08-5888-4472-a431-338969332732"
      },
      "execution_count": 18,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([[0.4235664 ],\n",
              "        [0.47960448],\n",
              "        [0.38275796],\n",
              "        [0.40072978]], dtype=float32), array([1., 0., 0., 1.], dtype=float32))"
            ]
          },
          "metadata": {},
          "execution_count": 18
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# prediction on all pad_txt_sequences\n",
        "pts_model.predict(pad_txt_sequences), target"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "_Rk7x9JpC79z",
        "outputId": "595a15ae-99a5-4213-d058-5b801b82021a"
      },
      "execution_count": 19,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([[0.4235664 ],\n",
              "        [0.47960448],\n",
              "        [0.2513317 ],\n",
              "        [0.44043738],\n",
              "        [0.35384014],\n",
              "        [0.38275796],\n",
              "        [0.47219223],\n",
              "        [0.551576  ],\n",
              "        [0.5537416 ],\n",
              "        [0.28070074],\n",
              "        [0.2640342 ],\n",
              "        [0.57914233],\n",
              "        [0.2899829 ],\n",
              "        [0.5729519 ],\n",
              "        [0.57209957],\n",
              "        [0.40072978],\n",
              "        [0.27548116]], dtype=float32),\n",
              " array([1., 0., 0., 1., 0., 0., 0., 1., 1., 0., 0., 1., 0., 1., 1., 1., 0.],\n",
              "       dtype=float32))"
            ]
          },
          "metadata": {},
          "execution_count": 19
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# prediction on a\n",
        "dv_model.predict(x_test_dv), y_test_dv"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "NjxLU-A2oPJj",
        "outputId": "84afec45-dc90-41dd-8fde-d329b8bb4555"
      },
      "execution_count": 20,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([[0.4545603],\n",
              "        [0.4545603],\n",
              "        [0.4545603],\n",
              "        [0.4545603]], dtype=float32), array([1., 0., 0., 1.], dtype=float32))"
            ]
          },
          "metadata": {},
          "execution_count": 20
        }
      ]
    },
    {
      "cell_type": "code",
      "source": [
        "# predict on all doc_vecs\n",
        "dv_model.predict(doc_vecs), target"
      ],
      "metadata": {
        "colab": {
          "base_uri": "https://localhost:8080/"
        },
        "id": "TDNdERfn5CfC",
        "outputId": "87ba8ac8-5a9e-4b11-da20-0c78844aa4e6"
      },
      "execution_count": 21,
      "outputs": [
        {
          "output_type": "execute_result",
          "data": {
            "text/plain": [
              "(array([[0.4545603],\n",
              "        [0.4545603],\n",
              "        [0.4545603],\n",
              "        [0.4545603],\n",
              "        [0.4545603],\n",
              "        [0.4545603],\n",
              "        [0.4545603],\n",
              "        [0.4545603],\n",
              "        [0.4545603],\n",
              "        [0.4545603],\n",
              "        [0.4545603],\n",
              "        [0.4545603],\n",
              "        [0.4545603],\n",
              "        [0.4545603],\n",
              "        [0.4545603],\n",
              "        [0.4545603],\n",
              "        [0.4545603]], dtype=float32),\n",
              " array([1., 0., 0., 1., 0., 0., 0., 1., 1., 0., 0., 1., 0., 1., 1., 1., 0.],\n",
              "       dtype=float32))"
            ]
          },
          "metadata": {},
          "execution_count": 21
        }
      ]
    },
    {
      "cell_type": "markdown",
      "source": [
        "#Comments"
      ],
      "metadata": {
        "id": "C1MNxzdA3wLC"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "##Regarding to data preprocessing\n",
        "\n",
        "There are many great NLP libraries out to implement to our projects. To get familiar with some of them, we tried to use several of them as NLTK, Gensim, re and more. Also, sklearn provides text preprocessing in some of its functions as we used in CountVectorizer, TfidfVectorizer and they performed slightly better than the manual approach we did.\n"
      ],
      "metadata": {
        "id": "hlxg8rD8Dwvw"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Regarding to topic modeling\n",
        "Topic modelling was a nice challenge for us. We thought working with LDA would be more interesting to discover the topics from the financial textual data. Also, visualization of the topics with pyLDAvis was helping us to understand it better. Additionally, we chose the number of components as 10 because we did not understand clearly how to pick it with the best way which is why we decided to keep it as default since we were dealing with a small amount of textual data."
      ],
      "metadata": {
        "id": "ki4Dg5VRD3zg"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Reparding to doc2vec & embedding\n",
        "\n",
        "Since we were dealing with text data that is sequential, we thought that implementing doc2vec might be smarter than working with word2vec since it is easier to focus on the semantic understanding of the text. Also, we took the vector_size as 60 because of the dimension of the embedded vector in order to compare them equally.\n"
      ],
      "metadata": {
        "id": "rxxHJgM2D9Y8"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Regarding to modelling\n",
        "\n",
        "Rather than focusing on the model architecture, we wanted to focus on how doc2vec and embedded vectors perform on the same model. As we have experienced, in our case embedded vectors are better to get a better semantic understanding from our textual data compare to doc2vec vectors as we can see on the predictions. "
      ],
      "metadata": {
        "id": "GyO1aY5qECEf"
      }
    },
    {
      "cell_type": "markdown",
      "source": [
        "## Final comments\n",
        "\n",
        "This notebook describes our own approaches to textual analysis assignment, which served as the second sub-assignment for the Deep Learning and Text Analysis in Finance course at the University of Passau.\n",
        "\n",
        "For this purpose, we applied different methods such as frequency, topic modelling, word embedding and more.\n",
        "\n",
        "Nevertheless, it was a great experience and a nice discovery of the text analysing world where lots of exciting new discoveries happening in recent years."
      ],
      "metadata": {
        "id": "wBXnQqD1EEwz"
      }
    }
  ]
}