project.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Emoji Prediction"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "Tetsumichi Umada (Telly)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#### Motivation\n",
    "The project idea originally comes from [DeepMoji](https://deepmoji.mit.edu), which translate given text messages into Emojis to describe their emotions, using millions of text messages and state-of-art algorithms. For this project, we predict emotions associated texts. Nowadays, people often send a text to each other. However, it is hard for people to understand and know a sender's emotions from the text. The classifier will further help to represent emotions associated with the text messages. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Datasets**: The data comes from [DeepMoji GitHub repository](https://github.com/bfelbo/DeepMoji/tree/master/data). All the data is in `pickle` format. Using python2 script, we unpickle the data and save it as a`txt` file. Then, we use python3 to apply machine learning algorithms. \n",
    "\n",
    "<br>\n",
    "The python script to extract `pickle` data is saved as `script.py`. The file name needs to be specified at the top of the script. Then, run the program with `python script.py > <filen_name>.txt` to save into a new txt file. And, each line contains text and its label, formatted with: `[<label>] <text>`. \n",
    "\n",
    "<br>\n",
    "We use `psychExp` datasets. Each instance is associated with seven kinds of emotions: joy, fear, anger, sadness, disgust, shame, guilt. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Read data from the text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def read_file(file_name): \n",
    "    data_list  = []\n",
    "    with open(file_name, 'r') as f: \n",
    "        for line in f: \n",
    "            line = line.strip() \n",
    "            label = ' '.join(line[1:line.find(\"]\")].strip().split())\n",
    "            text = line[line.find(\"]\")+1:].strip()\n",
    "            data_list.append([label, text])\n",
    "    return data_list "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "# file_name = \"olympic.txt\"\n",
    "file_name = \"data/psychExp.txt\"\n",
    "psychExp_txt = read_file(file_name)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "scrolled": true,
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The number of instances: 7480\n"
     ]
    }
   ],
   "source": [
    "print(\"The number of instances: {}\".format(len(psychExp_txt)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Data example: \n",
      "['1. 0. 0. 0. 0. 0. 0.', 'During the period of falling in love, each time that we met and especially when we had not met for a long time.']\n"
     ]
    }
   ],
   "source": [
    "print(\"Data example: \")\n",
    "print(psychExp_txt[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Features       \n",
    "From the given texts, we made ngrams with a range from 1 to 4. The feature also includes the any punctuations such as `!`, `?`, or `.`. If the data originally come form the Tweets, it contains hash tags (such as `#olympics2012` or `#olympiics`). They are kept as hashtags. \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "import re \n",
    "from collections import Counter"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def ngram(token, n): \n",
    "    output = []\n",
    "    for i in range(n-1, len(token)): \n",
    "        ngram = ' '.join(token[i-n+1:i+1])\n",
    "        output.append(ngram) \n",
    "    return output"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def create_feature(text, nrange=(1, 1)):\n",
    "    text_features = [] \n",
    "    text = text.lower() \n",
    "\n",
    "    # 1. treat alphanumeric characters as word tokens\n",
    "    # Since tweets contain #, we keep it as a feature\n",
    "    # Then, extract all ngram lengths\n",
    "    text_alphanum = re.sub('[^a-z0-9#]', ' ', text)\n",
    "    for n in range(nrange[0], nrange[1]+1): \n",
    "        text_features += ngram(text_alphanum.split(), n)\n",
    "    \n",
    "    # 2. treat punctuations as word token\n",
    "    text_punc = re.sub('[a-z0-9]', ' ', text)\n",
    "    text_features += ngram(text_punc.split(), 1)\n",
    "    \n",
    "    # 3. Return a dictinaory whose keys are the list of elements \n",
    "    # and their values are the number of times appearede in the list.\n",
    "    return Counter(text_features)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Counter({'i': 1, 'love': 1, 'you': 1, '!': 1})\n",
      "Counter({'aly': 1, 'wins': 1, 'the': 1, 'gold': 1, '#olympics': 1, '!!!!!!': 1, '#': 1})\n",
      "Counter({'aly': 1, 'wins': 1, 'the': 1, 'gold': 1, '#olympics': 1, 'aly wins': 1, 'wins the': 1, 'the gold': 1, 'gold #olympics': 1, '!!!!!!': 1, '#': 1})\n"
     ]
    }
   ],
   "source": [
    "print(create_feature(\"I love you!\"))\n",
    "print(create_feature(\" aly wins the gold!!!!!!  #olympics\"))\n",
    "print(create_feature(\" aly wins the gold!!!!!!  #olympics\", (1, 2)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Cnvert the lables and Add features to text data**: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "def convert_label(item, name): \n",
    "    items = list(map(float, item.split()))\n",
    "    label = \"\"\n",
    "    for idx in range(len(items)): \n",
    "        if items[idx] == 1: \n",
    "            label += name[idx] + \" \"\n",
    "    \n",
    "    return label.strip()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "emotions = [\"joy\", 'fear', \"anger\", \"sadness\", \"disgust\", \"shame\", \"guilt\"]\n",
    "\n",
    "X_all = []\n",
    "y_all = []\n",
    "for label, text in psychExp_txt:\n",
    "    y_all.append(convert_label(label, emotions))\n",
    "    X_all.append(create_feature(text, nrange=(1, 4)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "features example: \n",
      "Counter({'time': 2, 'we': 2, 'met': 2, 'during': 1, 'the': 1, 'period': 1, 'of': 1, 'falling': 1, 'in': 1, 'love': 1, 'each': 1, 'that': 1, 'and': 1, 'especially': 1, 'when': 1, 'had': 1, 'not': 1, 'for': 1, 'a': 1, 'long': 1, 'during the': 1, 'the period': 1, 'period of': 1, 'of falling': 1, 'falling in': 1, 'in love': 1, 'love each': 1, 'each time': 1, 'time that': 1, 'that we': 1, 'we met': 1, 'met and': 1, 'and especially': 1, 'especially when': 1, 'when we': 1, 'we had': 1, 'had not': 1, 'not met': 1, 'met for': 1, 'for a': 1, 'a long': 1, 'long time': 1, 'during the period': 1, 'the period of': 1, 'period of falling': 1, 'of falling in': 1, 'falling in love': 1, 'in love each': 1, 'love each time': 1, 'each time that': 1, 'time that we': 1, 'that we met': 1, 'we met and': 1, 'met and especially': 1, 'and especially when': 1, 'especially when we': 1, 'when we had': 1, 'we had not': 1, 'had not met': 1, 'not met for': 1, 'met for a': 1, 'for a long': 1, 'a long time': 1, 'during the period of': 1, 'the period of falling': 1, 'period of falling in': 1, 'of falling in love': 1, 'falling in love each': 1, 'in love each time': 1, 'love each time that': 1, 'each time that we': 1, 'time that we met': 1, 'that we met and': 1, 'we met and especially': 1, 'met and especially when': 1, 'and especially when we': 1, 'especially when we had': 1, 'when we had not': 1, 'we had not met': 1, 'had not met for': 1, 'not met for a': 1, 'met for a long': 1, 'for a long time': 1, ',': 1, '.': 1})\n"
     ]
    }
   ],
   "source": [
    "print(\"features example: \")\n",
    "print(X_all[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Label example:\n",
      "joy\n"
     ]
    }
   ],
   "source": [
    "print(\"Label example:\")\n",
    "print(y_all[0])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Classifiers \n",
    "The following classifiers are used for this project:\n",
    "+ `SVC`\n",
    "+ `LinearSVC`\n",
    "+ `RandomForestClassifier`\n",
    "+ `DecisionTreeClassifier`\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Data split**: The data is split into 80-20 ratio where 80% for the training and 20% for the testing. We keep 20% to make sure that we have enough amount of the data for testing. We used `train_test_split` to split the data. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.model_selection import train_test_split\n",
    "X_train, X_test, y_train, y_test = \\\n",
    "    train_test_split(X_all, y_all, test_size = 0.2, random_state = 123)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.metrics import accuracy_score\n",
    "\n",
    "def train_test(clf, X_train, X_test, y_train, y_test):\n",
    "    clf.fit(X_train, y_train)\n",
    "    train_acc = accuracy_score(y_train, clf.predict(X_train))\n",
    "    test_acc = accuracy_score(y_test, clf.predict(X_test))\n",
    "    \n",
    "#     print(\"Training acc: {}\".format(train_acc))\n",
    "#     print(\"Test acc    : {}\".format(test_acc))\n",
    "    \n",
    "    return train_acc, test_acc    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**DictVectorizer**: The above converts the labeled features into sparse vectors"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "from sklearn.feature_extraction import DictVectorizer\n",
    "vectorizer = DictVectorizer(sparse = True)\n",
    "X_train = vectorizer.fit_transform(X_train)\n",
    "X_test = vectorizer.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "| Classifier                | Training Accuracy | Test Accuracy |\n",
      "| ------------------------- | ----------------- | ------------- |\n",
      "| SVC                       |         0.9067513 |     0.4512032 |\n"
     ]
    },
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "| LinearSVC                 |         0.9988302 |     0.5768717 |\n",
      "| RandomForestClassifier    |         0.9988302 |     0.5541444 |\n",
      "| DecisionTreeClassifier    |         0.9988302 |     0.4625668 |\n"
     ]
    }
   ],
   "source": [
    "from sklearn.svm import SVC\n",
    "from sklearn.svm import LinearSVC\n",
    "from sklearn.ensemble import RandomForestClassifier\n",
    "from sklearn.tree import DecisionTreeClassifier\n",
    "\n",
    "# Classifiers \n",
    "svc = SVC()\n",
    "lsvc = LinearSVC(random_state=123)\n",
    "rforest = RandomForestClassifier(random_state=123)\n",
    "dtree = DecisionTreeClassifier()\n",
    "\n",
    "clifs = [svc, lsvc, rforest, dtree]\n",
    "\n",
    "# train and test them \n",
    "print(\"| {:25} | {} | {} |\".format(\"Classifier\", \"Training Accuracy\", \"Test Accuracy\"))\n",
    "print(\"| {} | {} | {} |\".format(\"-\"*25, \"-\"*17, \"-\"*13))\n",
    "for clf in clifs: \n",
    "    clf_name = clf.__class__.__name__\n",
    "    train_acc, test_acc = train_test(clf, X_train, X_test, y_train, y_test)\n",
    "    print(\"| {:25} | {:17.7f} | {:13.7f} |\".format(clf_name, train_acc, test_acc))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Results**: The training and test accuracies for each classifier are summarized in the table below.\n",
    "\n",
    "\n",
    "| Classifier                | Training Accuracy | Test Accuracy |\n",
    "| ------------------------- | ----------------- | ------------- |\n",
    "| SVC                       |         0.9067513 |     0.4512032 |\n",
    "| LinearSVC                 |         0.9988302 |     0.5768717 |\n",
    "| RandomForestClassifier    |         0.9988302 |     0.5541444 |\n",
    "| DecisionTreeClassifier    |         0.9988302 |     0.4625668 |\n",
    "\n",
    "\n",
    "\n",
    "By looking at the numbers, the support vector machine works good, meaning it's not overfitting and underfitting. The test accuracy is slightly higher than the other classifiers. The other classifiers are overfitting because the training accuracy is very higher than the test accuracy. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Model Tuning**: Since `LinearSVC` seems to be working good, we further try to tune parameters. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": true,
    "jupyter": {
     "outputs_hidden": true
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n",
      "C:\\Users\\FU_LAB\\anaconda3\\Lib\\site-packages\\sklearn\\svm\\_base.py:1244: ConvergenceWarning: Liblinear failed to converge, increase the number of iterations.\n",
      "  warnings.warn(\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Validation acc: 0.5608291911495573\n",
      "Training acc: 0.9988302139037433\n",
      "Test acc    : 0.5768716577540107\n",
      "Best parameter: {'C': 1, 'tol': 0.1}\n"
     ]
    }
   ],
   "source": [
    "from sklearn.model_selection import GridSearchCV\n",
    "\n",
    "parameters = {'C':[1, 2, 3, 5, 10, 15, 20, 30, 50, 70, 100], \n",
    "             'tol':[0.1, 0.01, 0.001, 0.0001, 0.00001]}\n",
    "\n",
    "lsvc = LinearSVC(random_state=123)\n",
    "grid_obj = GridSearchCV(lsvc, param_grid = parameters, cv=5)\n",
    "grid_obj.fit(X_train, y_train)\n",
    "\n",
    "print(\"Validation acc: {}\".format(grid_obj.best_score_))\n",
    "print(\"Training acc: {}\".format(accuracy_score(y_train, grid_obj.predict(X_train))))\n",
    "print(\"Test acc    : {}\".format(accuracy_score(y_test, grid_obj.predict(X_test))))\n",
    "print(\"Best parameter: {}\".format(grid_obj.best_params_))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Final model**: The final model for this project performs test accuracy 0.576872. Although we tried to tune to minimize the overfitting, it seems to be not working well. As we expected, the accuracy is low.... "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Error analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[[112  27  16  23  12  19  19]\n",
      " [ 20 113  11  11  12  17   9]\n",
      " [  9  18 157   3  11  16  10]\n",
      " [ 19  12  15 107  12  17  36]\n",
      " [  7   9   7   7 155  18   8]\n",
      " [ 17   9  16  13  21 137  10]\n",
      " [ 31  19  10  30  17  10  82]]\n"
     ]
    }
   ],
   "source": [
    "from sklearn.metrics import confusion_matrix\n",
    "matrix = confusion_matrix(y_test, grid_obj.predict(X_test))\n",
    "print(matrix)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAv0AAAJGCAYAAADIyzqiAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjguMCwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy81sbWrAAAACXBIWXMAAA9hAAAPYQGoP6dpAABfIklEQVR4nO3dd3hUZf7+8XsSkiEdQklRCCX0rgiISJG2iIig4ooFAZWiQBBEWXcRFImiFAEVYZGiiw2FtVCVoghKVXpooQjJBjSEHkLm+f3Bj/k6JjDJZJLJjO+X17ku5jlnztwZw/DJJ895jsUYYwQAAADAZ/l5OgAAAACAwkXRDwAAAPg4in4AAADAx1H0AwAAAD6Ooh8AAADwcRT9AAAAgI+j6AcAAAB8HEU/AAAA4ONKeDrAVWef7ebpCD6t4/wMT0fweccv/ubpCD6tdGCYpyP4vGPnT3o6gs8rbeX7GN5tT9pGT0fIVdbJg0X2WgFlqxTZa7kTnX4AAADAxxWbTj8AAADgElu2pxMUe3T6AQAAAB9Hpx8AAADezdg8naDYo9MPAAAA+DiKfgAAAMDHMb0HAAAA3s3G9B5n6PQDAAAAPo5OPwAAALya4UJep+j0AwAAAD6OTj8AAAC8G3P6naLTDwAAAPg4Ov0AAADwbszpd4pOPwAAAODj6PQDAADAu9myPZ2g2KPTDwAAAPg4Ov0AAADwbszpd4pOPwAAAODj6PQDAADAu7FOv1N0+gEAAAAfl++i//LlyxozZoyOHj1aGHkAAACAfDHGVmSbt8p30V+iRAm9/vrrys5maSQAAADAG7g0vaddu3ZavXq1m6MAAAAALrDZim7zUi5dyNupUyeNHDlSO3bs0M0336yQkBCH/XfffbdbwgEAAAAoOJeK/gEDBkiSJk6cmGOfxWJh6g8AAABQjLhU9Nu8+FcbAAAA8DFefIFtUSnwkp0XL150Rw4AAAAAhcSloj87O1svv/yybrjhBoWGhurgwYOSpH/961+aNWuWWwMCAAAA12XLLrrNS7lU9L/yyiuaM2eOxo8fr8DAQPt4vXr19O9//9tt4QAAAAAUnEtF/7x58zRjxgw99NBD8vf3t4/Xr19fe/bscVs4AAAAwCljK7rNS7lU9B87dkzx8fE5xm02m7KysgocCgAAAID7uLR6T506dfT9998rLi7OYfzTTz9Vo0aN3BIMAAAAyBNWlnTKpaL/xRdf1COPPKJjx47JZrPp888/V1JSkubNm6evvvrK3RmLhF/l2gpsfY/8bqgqv4hIXZiTqOydG+z7/es2U0CzDvK/saosIeE6P2mobMcP/d8JgkIV2OHvKlG9oSylysqcO63LO3/SpWUfShfPF/0X5AUeefpBtep0u+LiKyrzYqa2b9qpd8bN1JEDR+3H/HBsZa7PfevldzV/+sdFFdVrNbn1Jj359GOq27CWoqLL68lHErRi8SqHY6pWr6znRyWoyW03y8/PT/v2HNDTfZ7V8WOpHkrtPXoPelh33NlKleLjlHkxU79s2q4pY9/R4T98D/cb1kcd7mmr6Njyyrp0Wbu3JemtV2dox9ZdHkzuPZo1b6ynBvdV/YZ1FB1TXo/1fEpLvv5WklSiRAk9/68hate+leIq3ajTp8/qu9XrNHb0RP0vNc3Dyb1H42aN1PepR1SnQU2Vjy6np3oN17dL1tj3lykXqeH/GqTbWjdVWHiYNv24VWNHvq7DyUevc1ZcxfuL4sKl6T1dunTRxx9/rMWLF8tisWjUqFHavXu3vvzyS7Vv397dGYuEJbCkbMcPKXPRzGvstyr70B5lLn4/1/1+4ZHyi4hU5ldzdH5igjI/nqoSNW5SyfufKszYXq1hswb6fO5/9WSXp5Xw4LPyL+GvSfPHq2RQSfsxXRre67C9MnS8bDabVi/+zoPJvUdQcJB270zSi8+9muv+ipVu1Kdfz9GBfcl68O7HdWfL+zX1jRnKzLxUxEm90823NtInsz9Xr879NOCBoSrh76+3P5rk8D18+OBRvfaPSerRppf6dB2o40dT9NZHE1WqTCnPBfciwcFB2rljj0Y++3KOfUHBJVW/QW1NfP1ttWt5r/o8PEhV4ytp3kdveyCp9woKDtKenXv18sjXc93/1tzXdWNcrAY+Olzd2z6s47+m6L0FbykouGSux8MR728RYU6/UxZjjPF0CEk6+2w3T0ewC319YY5O/1WW0uUU8o8ZOTv9ufCv31wlH0zQuRf+7vFfO3Wcn+HR18+LUpER+nr7Qg3snqBfftqW6zGJs15ScGiwhjwwvIjTOXf84m+ejnBdyb/9kqPTP2Xma7p8+bKeGfCCB5PlTenAME9HcKpUmVJaueMrPd7tKW358ZdcjwkJDdb3+5ar//1DtGHt5iJOeH3Hzp/0dITr+l/GHodOf24a3lRXy1Yt0E112ujYrylFmC5vSluL9/fxnrSNDp3oSlUqaumPn+mu2x/Q/qQry3P7+flp3a5leuPlaVrwn/96Mq7X8YX3d0/aRk9HyFXmjhVF9lrWut7Z4C7wzblwbZaSwVem9jDPLE9CwkMkSadPnc51f+mypdW8bTN99eHioozlsywWi9p0uF3J+w9r7qfvaOOeVVq4/AO1v7ONp6N5rbCwK9/DGem5fw+XCCih7g931ZmMM9q7a39RRvvLCA8Pk81mU0ZG7v8PkD+B1gBJUmZmpn3MZrPpUtZl3dy0oYdS+Q7eXzey2Ypu81IuFf2lS5dWZGRkjq1MmTK64YYb1KpVK82ePdvdWb1LcJgC292vrB+XezqJ1xj84kD98tM2JScdynV/p/s76PzZ81qz5PuiDeajypSLVGhoiPoP6aM13/6gR+/rr2Vfr9T0uRPVtPnNno7nlZ4ZPUhbf/pFB5KSHcZvb9dca/cv14+HVuqhJ3towANDder34v/bN29jtQbqhdHD9PmnX+nsmXOejuMTDu47pGNHjuuZF55SeESYAgJK6IlBvVQ+qqzKRZXxdDyvx/uLouTShbyjRo3SK6+8ok6dOqlJkyYyxmjjxo1aunSpnnrqKSUnJ2vAgAG6fPmynnjiiRzPz8zMdPipVpKyLmfLWsI/x7FeyRqkoD4vyPa/X3VpBReb5sUzrwxW1VpVNKDb4Gsec9ffO2n5wm91KZNlYd3Bz+/Kz/wrlqzSe9M/kCTt3pGkm29poJ6P3a+f1hWvqSfF3fPjnlG12lXVp+vAHPs2/rBFD7brrVKRpdTtoS56bcZLevTOJ5X+26miD+qjSpQooXffmyg/P4ueGzbG03F8xuXL2Rrc5zmNnfwvbdi3UpcvX9b67zZqzTc/eDqaT+D9dR9jvPdOuUXFpaJ/7dq1Gjt2rPr37+8w/u6772r58uX67LPPVL9+fU2ZMiXXoj8xMVFjxjh+KI+8tYb+cVstV+IUL9aSCnp8lMyli7o491Wvvl1zURn68iC16NBcT3VP0ImU3OcUN2hST3HxFTVqwEtFnM53pf+WrqysLO3fe9BhfP++ZDXm18r5MmJsglp2uE2Pd3taaSkncuy/eOGijh46pqOHjmn7lp1a9MOHuqfnXZo99QMPpPU9JUqU0Mw5k1Qx7kbd2+UxuvxutnPbHnW74yGFhoUoIDBA6b+d0sdLZmvHL7s9Hc0n8P6iqLg0vWfZsmVq165djvG2bdtq2bJlkqQ777xTBw8ezHGMJI0cOVIZGRkO27Cm1V2JUrxYgxT0xGgp+7Iuzh4nXaYj7cwzYwerVafbNbjHMKUcvfYSkXc92El7fknS/l25f08h/7KyLmvb1p2qEl/JYbxy1TgdO1r8LoAsrp57ZajuuLOV+t0/RMfz+L5ZLBYFBgYWcrK/hqsFf5Wqcbq/a2+lp5/ydCSfdfbMOaX/dkpxlSuobsNaWrl0jfMnIc94fwuI1XuccqnTHxkZqS+//FJDhw51GP/yyy8VGRkpSTp37pzCwnJfpcBqtcpqtTqMnfX01J7AkvIrG21/6BcZJRNbSeb8WZlTJ6WgUPmVLitL+JWvz6/cDZIkc+aUzJlTVzr8T7woBVp18cPJVy7iLRl85Zizp736m6SwDBs3RO3vaavn+/xT58+eV2S50pKufPBduvh/S0YGhwarzV2tNO2l6Z6K6rWCQ4IUV7mi/XGFijeoVt0aykjP0PFjqZoxba6m/nu8NqzbrPVrN6pV29vUtmNLPXj34x5M7T2eTxymTt3aaWjvkTp/9rzKlLvy+XD2zFllXrykkkEl9XjCo1qz7AedTDupiNIRur9XN5WPKacVX65ycnZIUnBIsCpX+b/v4YpxN6pOvZo6lZ6h1JQ0zZr3puo1qK2HH+gvP39/lStfVpJ0Kj2DO8TnUXBIkCpWrmB/fGPFWNWsW10Z6RlKOfY/dezSVum/pev4sf+peq2qemHsMH27ZI1+WP2TB1N7D95fFBcuLdk5c+ZMDRgwQHfeeaeaNGkii8WiDRs2aPHixZo+fbr69u2rCRMmaMOGDfr447zNaff0kp3+VeooaMDYHONZm1ZeWXO/cRuVfCDnfPNLyz/SpRUfX/P5knRu3JMy6Tl/5V+UiuOSnde68dYrQ1/T4k+W2R/f/VBnDRnzlO5udL/OFeNf2xfHJTub3tZYH30xK8f4gg//q2efHiVJur/nPRqQ0EcxsVE6uP+QJr/2jlYsWV3ESZ0rjkt2bklZm+v4i0Ne0ZefLFGgNVDj3n5RdRvVVqnICGWkn9bOn3fr35Pnatcve4o4rXPFccnO5i2aaOHX83KMf/SfhXrj1WnatD335Tu7dX5U69bmXHbZ04rjkp1Nmt+keYvezTG+8KOvNHLwGD3y+APq89QjKlMuUif+d1L//WSx3pn4b2VlXfZAWu/ja+9vcV2y8+KWL4rstUredHeRvZY7ubxO/w8//KBp06YpKSlJxhjVrFlTgwYNUvPmzV0K4umi39cVx6Lf1xTHot+XFMei39cUx6Lf1xTHoh/ID4p+7y36XZreI0m33XabbrvtNndmAQAAAPKPadROuVz022w27d+/X2lpabL96UYFLVu2LHAwAAAAAO7hUtH/448/qmfPnjp8+LD+PDvIYrEoO5tlKgEAAIDiwqWiv3///mrcuLG+/vprxcTEyGKxuDsXAAAAkDfcF8kpl4r+ffv2acGCBYqPj3d3HgAAAABu5tLNuZo2bar9+/e7OwsAAACQf9ycyymXOv2DBg3SsGHDlJqaqnr16ikgIMBhf/369d0SDgAAAEDBuVT033vvvZKkPn365NjHhbwAAAAoUjbv7cAXFZeK/uTkZHfnAAAAAFBIXCr64+LiJEm7du3SkSNHdOnSJfs+i8Vi3w8AAAAUOi+ea19UXCr6Dx48qG7dumn79u2yWCz2tfqvLt3J9B4AAACg+HBp9Z4hQ4aocuXK+t///qfg4GDt2LFD3333nRo3bqzVq1e7OSIAAABwHTZb0W1eyqWif/369XrppZdUrlw5+fn5yd/fXy1atFBiYqIGDx7s7owAAACAV/ruu+/UpUsXxcbGymKxaNGiRdc8tl+/frJYLJo8ebLDeGZmpgYNGqSyZcsqJCREd999t3799dd85XCp6M/OzlZoaKgkqWzZsjp+/LikK3P9k5KSXDklAAAA4Jpi3Ok/d+6cGjRooGnTpl33uEWLFumnn35SbGxsjn0JCQlauHChPvroI61du1Znz57VXXfdla8p9S7N6a9bt662bdumKlWqqGnTpho/frwCAwM1Y8YMValSxZVTAgAAAD6nU6dO6tSp03WPOXbsmJ5++mktW7ZMnTt3dtiXkZGhWbNm6f3331e7du0kSR988IEqVKigb775Rh07dsxTDpeK/n/+8586d+6cJGns2LG66667dPvtt6tMmTL6+OOPXTklAAAA4BJjim4RmczMTGVmZjqMWa1WWa1Wl85ns9n0yCOP6Nlnn1WdOnVy7N+8ebOysrLUoUMH+1hsbKzq1q2rdevW5bnod2l6T8eOHdW9e3dJUpUqVbRr1y6dPHlSaWlpuuOOO1w5JQAAAFDsJSYmKiIiwmFLTEx0+XyvvfaaSpQocc3rYlNTUxUYGKjSpUs7jEdFRSk1NTXPr+NSpz83kZGR7joVAAAAkHdFuKrOyJEj9cwzzziMudrl37x5s958801t2bLFvvR9Xhlj8vUclzr9AAAAwF+R1WpVeHi4w+Zq0f/9998rLS1NFStWVIkSJVSiRAkdPnxYw4YNU6VKlSRJ0dHRunTpktLT0x2em5aWpqioqDy/FkU/AAAAvJuxFd3mRo888oi2bdumn3/+2b7Fxsbq2Wef1bJlyyRJN998swICArRixQr781JSUrRjxw41b948z6/ltuk9AAAAABydPXtW+/fvtz9OTk7Wzz//rMjISFWsWFFlypRxOD4gIEDR0dGqUaOGJCkiIkJ9+/bVsGHDVKZMGUVGRmr48OGqV6+efTWfvKDoBwAAAArJpk2b1KZNG/vjq9cD9OrVS3PmzMnTOSZNmqQSJUqoR48eunDhgtq2bas5c+bI398/zzko+gEAAODdivBC3vxq3bq1jDF5Pv7QoUM5xkqWLKmpU6dq6tSpLudgTj8AAADg4+j0AwAAwLu5+QJbX0SnHwAAAPBxdPoBAADg3YrxnP7igk4/AAAA4OPo9AMAAMC7MaffKTr9AAAAgI+j0w8AAADvxpx+p+j0AwAAAD6OTj8AAAC8G51+p4pN0d903nFPR/BpPz12g6cj+LzK0495OoJPO3v5oqcj+Dw/WTwdwedl2bI9HcGnnbyY4ekIQLFVbIp+AAAAwCWs3uMUc/oBAAAAH0enHwAAAN6NOf1O0ekHAAAAfBydfgAAAHg35vQ7RacfAAAA8HF0+gEAAODdmNPvFJ1+AAAAwMdR9AMAAAA+juk9AAAA8G5cyOsUnX4AAADAx9HpBwAAgHfjQl6n6PQDAAAAPo5OPwAAALwbnX6n6PQDAAAAPo5OPwAAALybMZ5OUOy51Om/4447dOrUqRzjp0+f1h133FHQTAAAAADcyKVO/+rVq3Xp0qUc4xcvXtT3339f4FAAAABAnjGn36l8Ff3btm2z/3nXrl1KTU21P87OztbSpUt1ww03uC8dAAAAgALLV9HfsGFDWSwWWSyWXKfxBAUFaerUqW4LBwAAADhFp9+pfBX9ycnJMsaoSpUq2rBhg8qVK2ffFxgYqPLly8vf39/tIQEAAAC4Ll9Ff1xcnCTJxk9TAAAAKC4MtakzLq3eM3fuXH399df2xyNGjFCpUqXUvHlzHT582G3hAAAAABScS0X/uHHjFBQUJElav369pk2bpvHjx6ts2bIaOnSoWwMCAAAA12WzFd3mpVxasvPo0aOKj4+XJC1atEj33XefnnzySd12221q3bq1O/MBAAAAKCCXOv2hoaH67bffJEnLly9Xu3btJEklS5bUhQsX3JcOAAAAcMaYotu8lEud/vbt2+vxxx9Xo0aNtHfvXnXu3FmStHPnTlWqVMmd+QAAAAAUkEud/rfeeku33nqrTpw4oc8++0xlypSRJG3evFkPPvigWwMCAAAAKBiXOv2lSpXStGnTcoyPGTOmwIEAAACAfPHiC2yLiktF/3fffXfd/S1btnQpDAAAAAD3c6noz22FHovFYv9zdna2y4EAAACAfKHT75RLc/rT09MdtrS0NC1dulS33HKLli9f7u6MAAAAAArApU5/REREjrH27dvLarVq6NCh2rx5c4GDAQAAAHli6PQ741Kn/1rKlSunpKQkd54SAAAAQAG51Onftm2bw2NjjFJSUvTqq6+qQYMGbgkGAAAA5IWxee9Ns4qKS0V/w4YNZbFYZP50V7JmzZrpvffec0swAAAAAO7hUtGfnJzs8NjPz0/lypVTyZIl3RIKAAAAyDNW73HKpaI/Li6uQC+amZmpzMxMhzGbscnP4tZLDAAAAADIxaJ/ypQpuY5bLBaVLFlS8fHxatmypfz9/XM9LjExMcfde8sGx6p86I2uxAEAAMBfGav3OOVS0T9p0iSdOHFC58+fV+nSpWWM0alTpxQcHKzQ0FClpaWpSpUqWrVqlSpUqJDj+SNHjtQzzzzjMNY0vq1rXwEAAACA63JpPs24ceN0yy23aN++ffrtt9/0+++/a+/evWratKnefPNNHTlyRNHR0Ro6dGiuz7darQoPD3fYmNoDAAAAl9hM0W1eyqVO/z//+U999tlnqlq1qn0sPj5eb7zxhu69914dPHhQ48eP17333uu2oAAAAABc41LRn5KSosuXL+cYv3z5slJTUyVJsbGxOnPmTMHSAQAAAM6weo9TLs2padOmjfr166etW7fax7Zu3aoBAwbojjvukCRt375dlStXdk9KAAAAAC5zqeifNWuWIiMjdfPNN8tqtcpqtapx48aKjIzUrFmzJEmhoaGaMGGCW8MCAAAAOdhsRbd5KZem90RHR2vFihVKSkpSUlKSjDGqWbOmatSoYT+mTZs2bgsJAAAAwHUuFf1X1ahRQzVq1FB2dra2b9+u9PR0lS5d2l3ZAAAAALiBS9N7EhIS7NN4srOz1apVK910002qUKGCVq9e7c58AAAAwPUZU3Sbl3Kp6F+wYIEaNGggSfryyy918OBB7dmzRwkJCXrhhRfcGhAAAADwVt999526dOmi2NhYWSwWLVq0yL4vKytLzz33nOrVq6eQkBDFxsbq0Ucf1fHjxx3OkZmZqUGDBqls2bIKCQnR3XffrV9//TVfOVwq+k+ePKno6GhJ0uLFi9WjRw9Vr15dffv21fbt2105JQAAAOCaYnwh77lz59SgQQNNmzYtx77z589ry5Yt+te//qUtW7bo888/1969e3X33Xc7HJeQkKCFCxfqo48+0tq1a3X27Fndddddys7OznMOl+b0R0VFadeuXYqJidHSpUv19ttv24P7+/u7ckoAAADA53Tq1EmdOnXKdV9ERIRWrFjhMDZ16lQ1adJER44cUcWKFZWRkaFZs2bp/fffV7t27SRJH3zwgSpUqKBvvvlGHTt2zFMOl4r+3r17q0ePHoqJiZHFYlH79u0lST/99JNq1qzpyikBAAAA19iKbq59ZmamMjMzHcauLmHvDhkZGbJYLCpVqpQkafPmzcrKylKHDh3sx8TGxqpu3bpat25dnot+l6b3jB49Wv/+97/15JNP6ocffrB/kf7+/nr++eddOSUAAABQ7CUmJioiIsJhS0xMdMu5L168qOeff149e/ZUeHi4JCk1NVWBgYE5VsiMiopSampqns/t8pKd9913X46xXr16uXo6AAAAwDWm6G6aNXLkSD3zzDMOY+7o8mdlZenvf/+7bDabfer89RhjZLFY8nz+PBf9U6ZM0ZNPPqmSJUtqypQp1z128ODBeQ4AAAAAeAt3TuW5KisrSz169FBycrJWrlxp7/JLV26Ke+nSpRz3w0pLS1Pz5s3z/Bp5LvonTZqkhx56SCVLltSkSZOueZzFYqHoBwAAQNEpwjn97na14N+3b59WrVqlMmXKOOy/+eabFRAQoBUrVqhHjx6SpJSUFO3YsUPjx4/P8+vkuehPTk7O9c8AAAAAcnf27Fnt37/f/jg5OVk///yzIiMjFRsbq/vuu09btmzRV199pezsbPs8/cjISAUGBioiIkJ9+/bVsGHDVKZMGUVGRmr48OGqV6+efTWfvMhz0f/nuUvXYrFYNGHChDwHAAAAAArCuLB+flHZtGmT2rRpY398tabu1auXRo8erS+++EKS1LBhQ4fnrVq1Sq1bt5Z0ZcZNiRIl1KNHD124cEFt27bVnDlz8rVUfp6L/q1btzo83rx5s7Kzs1WjRg1J0t69e+Xv76+bb745zy8OAAAA+LLWrVvLmGtPP7revqtKliypqVOnaurUqS7nyHPRv2rVKvufJ06cqLCwMM2dO9d+QUF6erp69+6t22+/3eUwAAAAQL558Zz+ouLSOv0TJkxQYmKiwxXEpUuX1tixY5naAwAAABQzLhX9p0+f1v/+978c42lpaTpz5kyBQwEAAAB5ZmxFt3kpl4r+bt26qXfv3lqwYIF+/fVX/frrr1qwYIH69u2r7t27uzsjAAAAgAJw6Y6806dP1/Dhw/Xwww8rKyvryolKlFDfvn31+uuvuzUgAAAAcF3M6XfKpaI/ODhYb7/9tl5//XUdOHBAxhjFx8crJCTE3fkAAAAAFJBLRf9VISEhql+/vruyAAAAAPlXjNfpLy5cmtMPAAAAwHtQ9AMAAAA+rkDTewAAAACP40Jep+j0AwAAAD6OTj8AAAC8mxffNKuo0OkHAAAAfBydfgAAAHg35vQ7RacfAAAA8HF0+gEAAODVDDfncopOPwAAAODjik2n/9dzJz0dwac1eu+ypyP4vOMHlng6gk8Lir3d0xF8XumgUE9H8Hk2k+3pCD7N30Iv8y+LOf1O8bcDAAAA8HHFptMPAAAAuIROv1N0+gEAAAAfR6cfAAAA3o078jpFpx8AAADwcXT6AQAA4N2Y0+8UnX4AAADAx9HpBwAAgFczdPqdotMPAAAA+DiKfgAAAMDHMb0HAAAA3o3pPU7R6QcAAAB8HJ1+AAAAeDcbN+dyhk4/AAAA4OPo9AMAAMC7MaffKTr9AAAAgI+j0w8AAADvRqffKTr9AAAAgI+j0w8AAACvZgydfmfo9AMAAAA+jk4/AAAAvBtz+p2i0w8AAAD4ODr9AAAA8G50+p2i0w8AAAD4ODr9AAAA8GqGTr9TdPoBAAAAH5evoj8rK0tVqlTRrl27CisPAAAAkD82U3Sbl8pX0R8QEKDMzExZLJbCygMAAADAzfI9vWfQoEF67bXXdPny5cLIAwAAAOSPrQg3L5XvC3l/+uknffvtt1q+fLnq1aunkJAQh/2ff/6528IBAAAAKLh8F/2lSpXSvffeWxhZAAAAABSCfBf9s2fPLowcAAAAgEtYstM5luwEAAAAfJxLN+dasGCBPvnkEx05ckSXLl1y2Ldlyxa3BAMAAADyhE6/U/nu9E+ZMkW9e/dW+fLltXXrVjVp0kRlypTRwYMH1alTp8LICAAAAKAA8l30v/3225oxY4amTZumwMBAjRgxQitWrNDgwYOVkZFRGBkBAACAa2PJTqfyXfQfOXJEzZs3lyQFBQXpzJkzkqRHHnlEH374oXvTAQAAACiwfM/pj46O1m+//aa4uDjFxcXpxx9/VIMGDZScnCxj8jafKjMzU5mZmQ5jxhju9AsAAIB8Y/Ue5/Ld6b/jjjv05ZdfSpL69u2roUOHqn379nrggQfUrVu3PJ0jMTFRERERDltmVnp+owAAAADIA4vJa3v+/7PZbLLZbCpR4sovCT755BOtXbtW8fHx6t+/vwIDA52eI7dO/40xDen0F6LyQaU8HcHn7dr9qacj+LSg2Ns9HcHnlQ4K9XQEnxdaoqSnI/i0jEvnPR3B5/1+Zp+nI+Qq/d7WRfZapT9bXWSv5U75nt7j5+cnP7//+wVBjx491KNHj3ydw2q1ymq1OoxR8AMAAACFw6Wbc33//fd6+OGHdeutt+rYsWOSpPfff19r1651azgAAADAGWMzRbbl13fffacuXbooNjZWFotFixYtcsxujEaPHq3Y2FgFBQWpdevW2rlzp8MxmZmZGjRokMqWLauQkBDdfffd+vXXX/OVI99F/2effaaOHTsqKChIW7dutU/TOXPmjMaNG5ff0wEAAAA+69y5c2rQoIGmTZuW6/7x48dr4sSJmjZtmjZu3Kjo6Gi1b9/evkKmJCUkJGjhwoX66KOPtHbtWp09e1Z33XWXsrOz85wj30X/2LFjNX36dM2cOVMBAQH28ebNm3M3XgAAABS9YrxOf6dOnTR27Fh17949xz5jjCZPnqwXXnhB3bt3V926dTV37lydP39e8+fPlyRlZGRo1qxZmjBhgtq1a6dGjRrpgw8+0Pbt2/XNN9/kOUe+i/6kpCS1bNkyx3h4eLhOnTqV39MBAAAAXiMzM1OnT5922P68QE1eJScnKzU1VR06dLCPWa1WtWrVSuvWrZMkbd68WVlZWQ7HxMbGqm7duvZj8iLfRX9MTIz279+fY3zt2rWqUqVKfk8HAAAAFIixFd2W29LziYmJLuVOTU2VJEVFRTmMR0VF2felpqYqMDBQpUuXvuYxeZHv1Xv69eunIUOG6L333pPFYtHx48e1fv16DR8+XKNGjcrv6QAAAACvMXLkSD3zzDMOY39elTK//ryKZV5uWpvfG9vmqejftm2b6tatKz8/P40YMUIZGRlq06aNLl68qJYtW8pqtWr48OF6+umn8/zCAAAAgFu4MNfeVbktPe+q6OhoSVe6+TExMfbxtLQ0e/c/Ojpaly5dUnp6ukO3Py0tTc2bN8/za+Vpek+jRo108uRJSVKVKlX0zDPP6MSJE9qwYYN+/PFHnThxQi+//HKeXxQAAAD4q6tcubKio6O1YsUK+9ilS5e0Zs0ae0F/8803KyAgwOGYlJQU7dixI19Ff546/aVKlVJycrLKly+vQ4cOyWazKSQkRI0bN87zCwEAAAB/NWfPnnW4HjY5OVk///yzIiMjVbFiRSUkJGjcuHGqVq2aqlWrpnHjxik4OFg9e/aUJEVERKhv374aNmyYypQpo8jISA0fPlz16tVTu3bt8pwjT0X/vffeq1atWikmJkYWi0WNGzeWv79/rscePHgwzy8OAAAAFJQpwuk9+bVp0ya1adPG/vjq9QC9evXSnDlzNGLECF24cEEDBw5Uenq6mjZtquXLlyssLMz+nEmTJqlEiRLq0aOHLly4oLZt22rOnDnXrMdzYzHG5OnWYkuXLtX+/fs1ePBgvfTSSw5B/mjIkCF5fvE/igit6tLzkDflg0p5OoLP27X7U09H8GlBsbd7OoLPKx0U6ukIPi+0RElPR/BpGZfOezqCz/v9zD5PR8jVyU6tiuy1yi5ZU2Sv5U55Xr3nb3/7m6Qra4UOGTLkmkU/AAAAUKSKcae/uMj3kp2zZ88ujBwAAAAACkm+i34AAACgOCnOc/qLi3zfkRcAAACAd6HTDwAAAK9Gp985Ov0AAACAj6PTDwAAAK9Gp985Ov0AAACAj6PTDwAAAO9mLJ5OUOzR6QcAAAB8HJ1+AAAAeDXm9DtHpx8AAADwcXT6AQAA4NWMjTn9ztDpBwAAAHwcnX4AAAB4Neb0O0enHwAAAPBxdPoBAADg1Qzr9DtFpx8AAADwcRT9AAAAgI9jeg8AAAC8GhfyOkenHwAAAPBxdPoBAADg1bg5l3N0+gEAAAAfR6cfAAAAXs0YTyco/opN0R8THOnpCD4tPfOMpyP4vKrVu3o6gk/7vVcdT0fwedU+OeLpCD7Pz+Lv6Qg+rXVkLU9HAIqtYlP0AwAAAK5gTr9zzOkHAAAAfBydfgAAAHg1Ov3O0ekHAAAAfBydfgAAAHg1Vu9xjk4/AAAA4OPo9AMAAMCrMaffOTr9AAAAgI+j0w8AAACvZgydfmfo9AMAAAA+jk4/AAAAvJqxeTpB8UenHwAAAPBxFP0AAACAj2N6DwAAALyajQt5naLTDwAAAPg4Ov0AAADwaizZ6RydfgAAAMDH0ekHAACAVzM2Ov3O0OkHAAAAfBydfgAAAHg1YzydoPij0w8AAAD4ODr9AAAA8GrM6XeOTj8AAADg4+j0AwAAwKtxR17n6PQDAAAAPs6lov+ll17S+fPnc4xfuHBBL730UoFDAQAAAHlljKXINm/lUtE/ZswYnT17Nsf4+fPnNWbMmAKHAgAAAOA+Ls3pN8bIYsn5k84vv/yiyMjIAocCAAAA8op1+p3LV9FfunRpWSwWWSwWVa9e3aHwz87O1tmzZ9W/f3+3hwQAAADgunwV/ZMnT5YxRn369NGYMWMUERFh3xcYGKhKlSrp1ltvdXtIAAAA4FpYvce5fBX9vXr1kiRVrlxZzZs3V0BAQKGEAgAAAOA+eS76T58+bf9zo0aNdOHCBV24cCHXY8PDwwueDAAAAMgDb15Vp6jkuegvVapUrhfv/tHVC3yzs7MLHAwAAACAe+S56F+1alVh5gAAAABQSPJc9Ldq1aowcwAAAAAuKa5Ldl6+fFmjR4/Wf/7zH6WmpiomJkaPPfaY/vnPf8rP78rtsowxGjNmjGbMmKH09HQ1bdpUb731lurUqePWLHku+rdt26a6devKz89P27Ztu+6x9evXL3AwAAAAwJu99tprmj59uubOnas6depo06ZN6t27tyIiIjRkyBBJ0vjx4zVx4kTNmTNH1atX19ixY9W+fXslJSUpLCzMbVnyXPQ3bNhQqampKl++vBo2bCiLxSKTy49VzOkHAABAUSquS3auX79eXbt2VefOnSVJlSpV0ocffqhNmzZJutLlnzx5sl544QV1795dkjR37lxFRUVp/vz56tevn9uy5LnoT05OVrly5ex/LojMzExlZmY6jNmMTX4WvwKdFwAAAChMudWxVqtVVqs1x7EtWrTQ9OnTtXfvXlWvXl2//PKL1q5dq8mTJ0u6UlOnpqaqQ4cODudq1aqV1q1b55miPy4uLtc/uyIxMVFjxoxxGCsTHKOyITcU6LwAAAD46ynKJTtzq2NffPFFjR49Osexzz33nDIyMlSzZk35+/srOztbr7zyih588EFJUmpqqiQpKirK4XlRUVE6fPiwW3Pn6+ZcV82bN++6+x999NHr7h85cqSeeeYZh7HGVdu4EgUAAAAoMrnVsbl1+SXp448/1gcffKD58+erTp06+vnnn5WQkKDY2Fj7TW8l5VgW/+oy+O7kUtF/9cKDq7KysnT+/HkFBgYqODjYadGf269AmNoDAAAAVxTlnP5rTeXJzbPPPqvnn39ef//73yVJ9erV0+HDh5WYmKhevXopOjpakuwr+1yVlpaWo/tfUC5V2unp6Q7b2bNnlZSUpBYtWujDDz90a0AAAADAG50/f96+NOdV/v7+stlskqTKlSsrOjpaK1assO+/dOmS1qxZo+bNm7s1i0ud/txUq1ZNr776qh5++GHt2bPHXacFAAAArquYLtOvLl266JVXXlHFihVVp04dbd26VRMnTlSfPn0kXZnWk5CQoHHjxqlatWqqVq2axo0bp+DgYPXs2dOtWdxW9EtXfnI5fvy4O08JAAAAeKWpU6fqX//6lwYOHKi0tDTFxsaqX79+GjVqlP2YESNG6MKFCxo4cKD95lzLly936xr9kmQxuS2278QXX3zh8NgYo5SUFE2bNk0VKlTQkiVL8h2kZvlb8v0c5F165hlPR/B5Vv8AT0fwaTvujXF+EAqk2idHPB3B54UGBHs6gk9rEHKjpyP4vM8Pf+H8IA9YF3Nvkb1W85TPiuy13MmlTv8999zj8NhisahcuXK64447NGHCBHfkAgAAAOAmLhX9Vy8+AAAAADytKNfp91YuFf1/Xpv0KovFopIlSyo+Pl5du3ZVZGRkgcIBAAAAKDiXiv6tW7dqy5Ytys7OVo0aNWSM0b59++Tv76+aNWvq7bff1rBhw7R27VrVrl3b3ZkBAAAAO+agOOfSOv1du3ZVu3btdPz4cW3evFlbtmzRsWPH1L59ez344IM6duyYWrZsqaFDh7o7LwAAAIB8cqnT//rrr2vFihUKDw+3j4WHh2v06NHq0KGDhgwZolGjRqlDhw5uCwoAAADkxog5/c641OnPyMhQWlpajvETJ07o9OnTkqRSpUrp0qVLBUsHAAAAoMBcnt7Tp08fLVy4UL/++quOHTumhQsXqm/fvvblPDds2KDq1au7MysAAAAAF7g0vefdd9/V0KFD9fe//12XL1++cqISJdSrVy9NmjRJklSzZk39+9//dl9SAAAAIBe2fN9q9q/HpaI/NDRUM2fO1KRJk3Tw4EEZY1S1alWFhobaj2nYsKG7MgIAAAAoAJeK/qtCQ0NVv359d2UBAAAA8s3GhbxOuTSnHwAAAID3KFCnHwAAAPA0lux0jk4/AAAA4OPo9AMAAMCr2TwdwAvQ6QcAAAB8HJ1+AAAAeDXm9DtHpx8AAADwcXT6AQAA4NWY0+8cnX4AAADAx9HpBwAAgFej0+8cnX4AAADAx9HpBwAAgFdj9R7n6PQDAAAAPo5OPwAAALyajUa/U3T6AQAAAB9Hpx8AAABezcacfqfo9AMAAAA+jqIfAAAA8HFM7wEAAIBXM54O4AXo9AMAAAA+rth0+o+ePeHpCD6thJ+/pyP4vLNZFz0dwafF/OeMpyP4vFMHlng6gs+rXf9hT0fwad+c3OHpCPAQm6cDeAE6/QAAAICPKzadfgAAAMAVNgtLdjpDpx8AAADwcXT6AQAA4NVYvcc5Ov0AAACAj6PTDwAAAK/G6j3O0ekHAAAAfBydfgAAAHg1G4v3OEWnHwAAAPBxdPoBAADg1Wyi1e8MnX4AAADAx9HpBwAAgFdjnX7n6PQDAAAAPo5OPwAAALwaq/c4R6cfAAAA8HEU/QAAAICPY3oPAAAAvJrN0wG8AJ1+AAAAwMfR6QcAAIBXY8lO5+j0AwAAAD6OTj8AAAC8Gkt2OkenHwAAAPBxdPoBAADg1Vi9xzk6/QAAAICPo9MPAAAAr0an3zk6/QAAAICPo9MPAAAAr2ZYvccpOv0AAABAITl27JgefvhhlSlTRsHBwWrYsKE2b95s32+M0ejRoxUbG6ugoCC1bt1aO3fudHsOin4AAAB4NVsRbvmRnp6u2267TQEBAVqyZIl27dqlCRMmqFSpUvZjxo8fr4kTJ2ratGnauHGjoqOj1b59e505c8aFd+La8j29Jzk5WZUrV3ZrCAAAAMDXvPbaa6pQoYJmz55tH6tUqZL9z8YYTZ48WS+88IK6d+8uSZo7d66ioqI0f/589evXz21Z8t3pj4+PV5s2bfTBBx/o4sWLbgsCAAAAuKIoO/2ZmZk6ffq0w5aZmZlrri+++EKNGzfW/fffr/Lly6tRo0aaOXOmfX9ycrJSU1PVoUMH+5jValWrVq20bt0697w5/1++i/5ffvlFjRo10rBhwxQdHa1+/fppw4YNbg0FAAAAFEeJiYmKiIhw2BITE3M99uDBg3rnnXdUrVo1LVu2TP3799fgwYM1b948SVJqaqokKSoqyuF5UVFR9n3uku+iv27dupo4caKOHTum2bNnKzU1VS1atFCdOnU0ceJEnThxwq0BAQAAgOsxRbiNHDlSGRkZDtvIkSNzzWWz2XTTTTdp3LhxatSokfr166cnnnhC77zzjsNxFovj8kPGmBxjBeXyhbwlSpRQt27d9Mknn+i1117TgQMHNHz4cN1444169NFHlZKS4s6cAAAAgMdZrVaFh4c7bFarNddjY2JiVLt2bYexWrVq6ciRI5Kk6OhoScrR1U9LS8vR/S8ol4v+TZs2aeDAgYqJidHEiRM1fPhwHThwQCtXrtSxY8fUtWtXd+YEAAAAcmWzFN2WH7fddpuSkpIcxvbu3au4uDhJUuXKlRUdHa0VK1bY91+6dElr1qxR8+bNC/y+/FG+V++ZOHGiZs+eraSkJN15552aN2+e7rzzTvn5Xfn5oXLlynr33XdVs2ZNtwYFAAAAvMnQoUPVvHlzjRs3Tj169NCGDRs0Y8YMzZgxQ9KVaT0JCQkaN26cqlWrpmrVqmncuHEKDg5Wz5493Zol30X/O++8oz59+qh37972X0n8WcWKFTVr1qwChwMAAAC81S233KKFCxdq5MiReumll1S5cmVNnjxZDz30kP2YESNG6MKFCxo4cKDS09PVtGlTLV++XGFhYW7NYjHGGLeeMQ8yMzNzLG0UHVXP7Rcs4P+U8PP3dASfd9mW7ekIQIGcOrDE0xF8Xu36D3s6gk/73/l0T0fweafPHfR0hFxNqlh0f7eGHvmgyF7LnfLd6ZekU6dOadasWdq9e7csFotq1aqlvn37KiIiIk/PT0xM1JgxYxyDlIhQYEApV+IAAAAAuI58X8i7adMmVa1aVZMmTdLvv/+ukydPatKkSapataq2bNmSp3PkttRRQIm8/cAAAAAA/FFR3pzLW+W70z906FDdfffdmjlzpkqUuPL0y5cv6/HHH1dCQoK+++47p+ewWq05ljZiag8AAABQOPJd9G/atMmh4JeurNk/YsQINW7c2K3hAAAAAGeK/AJVL5Tv6T3h4eH2Gwr80dGjR91+lTEAAACAgst3p/+BBx5Q37599cYbb6h58+ayWCxau3atnn32WT344IOFkREAAAC4pvzeNOuvKN9F/xtvvCGLxaJHH31Uly9fljFGgYGBGjBggF599dXCyAgAAACgAPJd9AcGBurNN99UYmKiDhw4IGOM4uPjFRwcXBj5AAAAgOvy5lV1ikqeiv7u3btrzpw5Cg8PV/fu3a97bGhoqOrUqaP+/fvned1+AAAAAIUnT0V/RESEfUlNZ4V8Zmampk+frh9++EFffPFFwRMCAAAA18HqPc7lqeifPXt2rn++ll27dumWW25xPRUAAAAAt8n3nP68qFGjhtatW1cYpwYAAAAc2Oj1O5Xvdfrzwt/fXw0aNCiMUwMAAADIp0Lp9AMAAABFhdV7nCuUTj8AAACA4oNOPwAAALwaM/qdo9MPAAAA+DiKfgAAAMDHMb0HAAAAXo0LeZ2j0w8AAAD4ODr9AAAA8Go2i6cTFH90+gEAAAAfR6cfAAAAXs3Gop1O0ekHAAAAfBydfgAAAHg1+vzO0ekHAAAAfBydfgAAAHg11ul3jk4/AAAA4OPo9AMAAMCrsXqPc3T6AQAAAB9Hpx8AAABejT6/c3T6AQAAAB9Hpx8AAABejdV7nKPTDwAAAPg4Ov0AAADwaqze4xydfgAAAMDH0ekHAACAV6PP7xydfgAAAMDHFZtOf3RwpKcj+LSTFzM8HcHnlQ+K8HQEn3YxO8vTEXxeo4a9PR3B5237T19PR/BpN9w7ydMRgGKr2BT9AAAAgCtYstM5pvcAAAAAPo5OPwAAALya4VJep+j0AwAAAD6OTj8AAAC8GnP6naPTDwAAAPg4Ov0AAADwajbm9DtFpx8AAADwcXT6AQAA4NXo8ztHpx8AAADwcXT6AQAA4NWY0+8cnX4AAADAx9HpBwAAgFdjnX7nXOr0X7hwQefPn7c/Pnz4sCZPnqzly5e7LRgAAAAA93Cp6O/atavmzZsnSTp16pSaNm2qCRMmqGvXrnrnnXfcGhAAAAC4HlOE/3krl4r+LVu26Pbbb5ckLViwQFFRUTp8+LDmzZunKVOmuDUgAAAAgIJxaU7/+fPnFRYWJklavny5unfvLj8/PzVr1kyHDx92a0AAAADgepjT75xLnf74+HgtWrRIR48e1bJly9ShQwdJUlpamsLDw90aEAAAAEDBuFT0jxo1SsOHD1elSpXUtGlT3XrrrZKudP0bNWrk1oAAAAAACsal6T333XefWrRooZSUFDVo0MA+3rZtW3Xr1s1t4QAAAABnvPkC26Li8s25oqOj1ahRI/n5+en06dNatGiRwsLCVLNmTXfmAwAAAHxCYmKiLBaLEhIS7GPGGI0ePVqxsbEKCgpS69attXPnTre/tktFf48ePTRt2jRJV9bsb9y4sXr06KH69evrs88+c2tAAAAA4HpsRbi5auPGjZoxY4bq16/vMD5+/HhNnDhR06ZN08aNGxUdHa327dvrzJkzBXi1nFwq+r/77jv7kp0LFy6UMUanTp3SlClTNHbsWLcGBAAAALzZ2bNn9dBDD2nmzJkqXbq0fdwYo8mTJ+uFF15Q9+7dVbduXc2dO1fnz5/X/Pnz3ZrBpaI/IyNDkZGRkqSlS5fq3nvvVXBwsDp37qx9+/a5NSAAAABwPTZjimzLzMzU6dOnHbbMzMzr5nvqqafUuXNntWvXzmE8OTlZqamp9pUwJclqtapVq1Zat26dW98jl4r+ChUqaP369Tp37pyWLl1qD5qenq6SJUu6NSAAAABQXCQmJioiIsJhS0xMvObxH330kbZs2ZLrMampqZKkqKgoh/GoqCj7PndxafWehIQEPfTQQwoNDVXFihXVunVrSVem/dSrV8+d+QAAAIDrKsq1e0aOHKlnnnnGYcxqteZ67NGjRzVkyBAtX778uo1xi8Xi8NgYk2OsoFwq+gcOHKgmTZro6NGjat++vfz8rvzCoEqVKszpBwAAgM+yWq3XLPL/bPPmzUpLS9PNN99sH8vOztZ3332nadOmKSkpSdKVjn9MTIz9mLS0tBzd/4JyqeiXpMaNG6t+/fpKTk5W1apVVaJECXXu3Nmd2QAAAACnbMV0nf62bdtq+/btDmO9e/dWzZo19dxzz6lKlSqKjo7WihUr7De4vXTpktasWaPXXnvNrVlcKvrPnz+vQYMGae7cuZKkvXv3qkqVKho8eLBiY2P1/PPPuzUkAAAA4G3CwsJUt25dh7GQkBCVKVPGPp6QkKBx48apWrVqqlatmsaNG6fg4GD17NnTrVlcupB35MiR+uWXX7R69WqH+Unt2rXTxx9/7LZwAAAAgDOmCP9ztxEjRighIUEDBw5U48aNdezYMS1fvlxhYWFufR2XOv2LFi3Sxx9/rGbNmjlcZFC7dm0dOHDAbeEAAAAAX7J69WqHxxaLRaNHj9bo0aML9XVdKvpPnDih8uXL5xg/d+6c2680BgAAAK6nIHfK/atwaXrPLbfcoq+//tr++GqhP3PmTN16663uSQYAAADALVzq9CcmJupvf/ubdu3apcuXL+vNN9/Uzp07tX79eq1Zs8bdGQEAAIBrKq6r9xQnLnX6mzdvrh9++EHnz59X1apVtXz5ckVFRWn9+vUO65ACAAAA8DyX1+mvV6+efclOAAAAwFMKY1UdX+Ny0W+z2bR//36lpaXJZnO8fKJly5YFDgYAAADAPVwq+n/88Uf17NlThw8fljGOP1lZLBZlZ2df9/mZmZnKzMx0GDPGJovFpdlGAAAA+Atj9R7nXKqy+/fvr8aNG2vHjh36/ffflZ6ebt9+//13p89PTExURESEw5Z+4X+uRAEAAADghEud/n379mnBggWKj4936UVHjhypZ555xmGsYWWmBAEAAACFwaWiv2nTptq/f7/LRb/VapXVanUYY2oPAAAAXPHn6ebIyaWif9CgQRo2bJhSU1NVr149BQQEOOyvX7++W8IBAAAAKDiXiv57771XktSnTx/7mMVikTEmTxfyAgAAAO7Czbmcc6noT05OdncOAAAAAIXEpaI/Li7O3TkAAAAAl7Bkp3N5Lvq/+OKLPJ/07rvvdikMAAAAAPfLc9F/zz33ODy+Oof/j4+vYk4/AAAAiophTr9TeV4n02az2bfly5erYcOGWrJkiU6dOqWMjAwtXrxYN910k5YuXVqYeQEAAADkk0tz+hMSEjR9+nS1aNHCPtaxY0cFBwfrySef1O7du90WEAAAALgeVu9xzqU7Yh04cEARERE5xiMiInTo0KGCZgIAAADgRi4V/bfccosSEhKUkpJiH0tNTdWwYcPUpEkTt4UDAAAAnDHGFNnmrVwq+t977z2lpaUpLi5O8fHxio+PV8WKFZWSkqJZs2a5OyMAAACAAnBpTn98fLy2bdumFStWaM+ePTLGqHbt2mrXrp3DKj4AAABAYWOdfudcKvqlK0t0dujQQR06dHBnHgAAAABu5nLRf+7cOa1Zs0ZHjhzRpUuXHPYNHjy4wMEAAACAvGCdfudcKvq3bt2qO++8U+fPn9e5c+cUGRmpkydPKjg4WOXLl6foBwAAAIoRly7kHTp0qLp06aLff/9dQUFB+vHHH3X48GHdfPPNeuONN9ydEQAAALgmm0yRbd7KpaL/559/1rBhw+Tv7y9/f39lZmaqQoUKGj9+vP7xj3+4OyMAAACAAnCp6A8ICLCv0hMVFaUjR45IunJzrqt/BgAAAFA8uDSnv1GjRtq0aZOqV6+uNm3aaNSoUTp58qTef/991atXz90ZAQAAgGvy5ptmFRWXOv3jxo1TTEyMJOnll19WmTJlNGDAAJ04cULvvvuuWwMCAAAAKBiXOv116tSx/0RVrlw5vf3221q4cKFq166thg0bujMfAAAAcF3efIFtUXGp09+1a1fNmzdPknTq1Ck1a9ZMEydO1D333KN33nnHrQEBAAAAFIxLRf+WLVt0++23S5IWLFigqKgoHT58WPPmzdOUKVPcGhAAAAC4HlOE/3krl4r+8+fPKywsTJK0fPlyde/eXX5+fmrWrJkOHz7s1oAAAAAACsaloj8+Pl6LFi3S0aNHtWzZMnXo0EGSlJaWpvDwcLcGBAAAAK7HZkyRbd7KpaJ/1KhRGj58uCpVqqSmTZvq1ltvlXSl69+oUSO3BgQAAABQMC6t3nPfffepRYsWSklJUYMGDezjbdu2Vbdu3dwWDgAAAHDGe/vvRcelol+SoqOjFR0d7TDWpEmTAgcCAAAA4F4uF/0AAABAccA6/c65NKcfAAAAgPeg0w8AAACvRqffOTr9AAAAgI+j0w8AAACvZrx4/fyiQqcfAAAA8HF0+gEAAODVmNPvXLEp+qsGlfd0BJ8W4Ofv6Qg+L+3CKU9H8GmNIip7OoLPO3TxpKcj+Lwb7p3k6Qg+bWVkLU9HAIqtYlP0AwAAAK4wdPqdYk4/AAAA4OMo+gEAAAAfx/QeAAAAeDWW7HSOTj8AAADg4+j0AwAAwKuxZKdzdPoBAAAAH0enHwAAAF6NOf3O0ekHAAAAfBydfgAAAHg15vQ7R6cfAAAA8HF0+gEAAODVDJ1+p+j0AwAAAD6Ooh8AAABezWZMkW35kZiYqFtuuUVhYWEqX7687rnnHiUlJTkcY4zR6NGjFRsbq6CgILVu3Vo7d+5059sjiaIfAAAAKBRr1qzRU089pR9//FErVqzQ5cuX1aFDB507d85+zPjx4zVx4kRNmzZNGzduVHR0tNq3b68zZ864NQtz+gEAAODViuuc/qVLlzo8nj17tsqXL6/NmzerZcuWMsZo8uTJeuGFF9S9e3dJ0ty5cxUVFaX58+erX79+bstCpx8AAADIo8zMTJ0+fdphy8zMzNNzMzIyJEmRkZGSpOTkZKWmpqpDhw72Y6xWq1q1aqV169a5NTdFPwAAALxaUc7pT0xMVEREhMOWmJjoNKMxRs8884xatGihunXrSpJSU1MlSVFRUQ7HRkVF2fe5i8tF//fff6+HH35Yt956q44dOyZJev/997V27Vq3hQMAAACKk5EjRyojI8NhGzlypNPnPf3009q2bZs+/PDDHPssFovDY2NMjrGCcqno/+yzz9SxY0cFBQVp69at9l9pnDlzRuPGjXNrQAAAAOB6TBH+Z7VaFR4e7rBZrdbr5hs0aJC++OILrVq1SjfeeKN9PDo6WpJydPXT0tJydP8LyqWif+zYsZo+fbpmzpypgIAA+3jz5s21ZcsWt4UDAAAAvJUxRk8//bQ+//xzrVy5UpUrV3bYX7lyZUVHR2vFihX2sUuXLmnNmjVq3ry5W7O4tHpPUlKSWrZsmWM8PDxcp06dKmgmAAAAwOs99dRTmj9/vv773/8qLCzM3tGPiIhQUFCQLBaLEhISNG7cOFWrVk3VqlXTuHHjFBwcrJ49e7o1i0tFf0xMjPbv369KlSo5jK9du1ZVqlRxRy4AAAAgT/J706yi8s4770iSWrdu7TA+e/ZsPfbYY5KkESNG6MKFCxo4cKDS09PVtGlTLV++XGFhYW7N4lLR369fPw0ZMkTvvfeeLBaLjh8/rvXr12v48OEaNWqUWwMCAAAA3sjk4YcRi8Wi0aNHa/To0YWaxaWif8SIEcrIyFCbNm108eJFtWzZUlarVcOHD9fTTz/t7owAAADANRXXm3MVJy7fkfeVV17RCy+8oF27dslms6l27doKDQ11ZzYAAAAAbuBy0S9JwcHBaty4sbuyAAAAAPlWXOf0FycuFf0XL17U1KlTtWrVKqWlpclmsznsZ9lOAAAAoPhwqejv06ePVqxYofvuu09NmjRx+x3DAAAAgLxiTr9zLhX9X3/9tRYvXqzbbrvN3XkAAAAAuJlLRf8NN9zg9rVDAQAAAFcYY3N+0F+cnytPmjBhgp577jkdPnzY3XkAAAAAuJlLnf7GjRvr4sWLqlKlioKDgxUQEOCw//fff3dLOAAAAMAZG3P6nXKp6H/wwQd17NgxjRs3TlFRUVzICwAAABRjLhX969at0/r169WgQQN35wEAAADyxbBOv1MuzemvWbOmLly44O4sAAAAAAqBS0X/q6++qmHDhmn16tX67bffdPr0aYcNAAAAKCo2mSLbvJVL03v+9re/SZLatm3rMG6MkcViUXZ29nWfn5mZqczMTIcxm7HJz+LSzyAAAAAArsOlon/VqlUFetHExESNGTPGYaxKWFVVjYgv0HkBAADw18OcfucsxgPvUm6d/u6176PTX4iOZP7m6Qg+L+3CKU9H8GmNIip7OoLPO3TxpKcj+Lz0zDOejuDTVkbW8nQEn9fw8BeejpCrG0rXKbLXOpa+s8hey51c6vRfdf78eR05ckSXLl1yGK9fv/51n2e1WmW1Wh3GKPgBAADgChudfqdcKvpPnDih3r17a8mSJbnudzanHwAAAEDRcam9npCQoPT0dP34448KCgrS0qVLNXfuXFWrVk1ffFE8f+0DAAAA/FW51OlfuXKl/vvf/+qWW26Rn5+f4uLi1L59e4WHhysxMVGdO3d2d04AAAAgV8aLl9IsKi51+s+dO6fy5ctLkiIjI3XixAlJUr169bRlyxb3pQMAAABQYC4V/TVq1FBSUpIkqWHDhnr33Xd17NgxTZ8+XTExMW4NCAAAAFyPMabINm/l0vSehIQEpaSkSJJefPFFdezYUf/5z38UGBioOXPmuDMfAAAAgAJyqeh/6KGH7H9u1KiRDh06pD179qhixYoqW7as28IBAAAAztiY0+9Ugdbpvyo4OFg33XSTO04FAAAAwM1cKvqzs7M1Z84cffvtt0pLS5PNZnPYv3LlSreEAwAAAJzx5rn2RcWlon/IkCGaM2eOOnfurLp168pisbg7FwAAAAA3cano/+ijj/TJJ5/ozjvvdHceAAAAIF9sdPqdcmnJzsDAQMXHx7s7CwAAAIBC4FLRP2zYML355pvMnwIAAIDHsU6/c3me3tO9e3eHxytXrtSSJUtUp04dBQQEOOz7/PPP3ZMOAAAAQIHlueiPiIhweNytWze3hwEAAADyi3X6nctz0T979mz7ny9cuCCbzaaQkBBJ0qFDh7Ro0SLVqlVLHTt2dH9KAAAAAC5zaU5/165d9f7770uSTp06pWbNmmnChAm655579M4777g1IAAAAHA9zOl3zqWif8uWLbr99tslSQsWLFBUVJQOHz6sefPmacqUKW4NCAAAAKBgXFqn//z58woLC5MkLV++XN27d5efn5+aNWumw4cPuzUgAAAAcD2s0++cS53++Ph4LVq0SEePHtWyZcvUoUMHSVJaWprCw8PdGhAAAABAwbhU9I8aNUrDhw9XpUqV1LRpU916662SrnT9GzVq5NaAAAAAAArGpek99913n1q0aKGUlBQ1aNDAPt62bVuW8gQAAECRMizZ6ZRLRb8kRUdHKzo62mGsSZMmBQ4EAAAAwL1cLvoBAACA4oALeZ1zaU4/AAAAAO9Bpx8AAABezZtvmlVU6PQDAAAAPo5OPwAAALwaq/c4R6cfAAAA8HF0+gEAAODVmNPvHJ1+AAAAwMfR6QcAAIBXo9PvHJ1+AAAAwMfR6QcAAIBXo8/vHJ1+AAAAwMdZDJOg8i0zM1OJiYkaOXKkrFarp+P4JN7jwsX7W/h4jwsX72/h4z0uXLy/KGoU/S44ffq0IiIilJGRofDwcE/H8Um8x4WL97fw8R4XLt7fwsd7XLh4f1HUmN4DAAAA+DiKfgAAAMDHUfQDAAAAPo6i3wVWq1UvvvgiF94UIt7jwsX7W/h4jwsX72/h4z0uXLy/KGpcyAsAAAD4ODr9AAAAgI+j6AcAAAB8HEU/AAAA4OMo+gEAAAAfR9EPt2rdurUSEhIkSZUqVdLkyZM9muevxBijJ598UpGRkbJYLPr55589HQn/36FDhxz+n6xevVoWi0WnTp3yaK7i7rHHHtM999zj6Rh/CRaLRYsWLfJ0DK/E9ym8RQlPB4Dv2rhxo0JCQjwdQ9KVoqty5craunWrGjZs6Ok4hWLp0qWaM2eOVq9erSpVqqhs2bKejoT/r0KFCkpJSbnm/5M5c+YoISGBHwL+5M033xQLzAGAe1D0FxNZWVkKCAjwdAy3KleunKcj/KUcOHBAMTExat68eaG9xqVLlxQYGFho5/dV/v7+io6O9nQMrxMREeHpCADgM/5y03uWLl2qFi1aqFSpUipTpozuuusuHThwQNL//Qr+888/V5s2bRQcHKwGDRpo/fr1DueYOXOmKlSooODgYHXr1k0TJ05UqVKlHI758ssvdfPNN6tkyZKqUqWKxowZo8uXL9v3WywWTZ8+XV27dlVISIjGjh1b6F+7u507d06PPvqoQkNDFRMTowkTJjjs//P0ntGjR6tixYqyWq2KjY3V4MGD7ftSUlLUuXNnBQUFqXLlypo/f77D8/88PUKSTp06JYvFotWrV0uS0tPT9dBDD6lcuXIKCgpStWrVNHv2bElS5cqVJUmNGjWSxWJR69at3f5+eNJjjz2mQYMG6ciRI7JYLKpUqZKMMRo/fryqVKmioKAgNWjQQAsWLLA/Jzs7W3379lXlypUVFBSkGjVq6M0338xx3nvuuUeJiYmKjY1V9erVi/pLKxbOnDmjhx56SCEhIYqJidGkSZMcprLlNjWiVKlSmjNnjqTcv3+vWr16tXr37q2MjAxZLBZZLBaNHj26UL8eb/HHaROZmZkaPHiwypcvr5IlS6pFixbauHGjpCtT2+Lj4/XGG284PH/Hjh3y8/Ozf8b7kgULFqhevXoKCgpSmTJl1K5dO507d04bN25U+/btVbZsWUVERKhVq1basmWLw3P37dunli1bqmTJkqpdu7ZWrFjhsD+v/xauW7dOLVu2VFBQkCpUqKDBgwfr3Llz9v1vv/22qlWrppIlSyoqKkr33Xef0/zFmbPMb7zxhmJiYlSmTBk99dRTysrKsu/74IMP1LhxY4WFhSk6Olo9e/ZUWlqaff/VKX/Lli1To0aNFBQUpDvuuENpaWlasmSJatWqpfDwcD344IM6f/68/XnOPucBB+YvZsGCBeazzz4ze/fuNVu3bjVdunQx9erVM9nZ2SY5OdlIMjVr1jRfffWVSUpKMvfdd5+Ji4szWVlZxhhj1q5da/z8/Mzrr79ukpKSzFtvvWUiIyNNRESE/TWWLl1qwsPDzZw5c8yBAwfM8uXLTaVKlczo0aPtx0gy5cuXN7NmzTIHDhwwhw4dKuq3osAGDBhgbrzxRrN8+XKzbds2c9ddd5nQ0FAzZMgQY4wxcXFxZtKkScYYYz799FMTHh5uFi9ebA4fPmx++uknM2PGDPu52rVrZxo2bGh+/PFHs3nzZtOqVSsTFBRkf/7V/zdbt261Pyc9Pd1IMqtWrTLGGPPUU0+Zhg0bmo0bN5rk5GSzYsUK88UXXxhjjNmwYYORZL755huTkpJifvvtt8J+e4rUqVOnzEsvvWRuvPFGk5KSYtLS0sw//vEPU7NmTbN06VJz4MABM3v2bGO1Ws3q1auNMcZcunTJjBo1ymzYsMEcPHjQfPDBByY4ONh8/PHH9vP26tXLhIaGmkceecTs2LHDbN++3VNfokc9/vjjJi4uznzzzTdm+/btplu3biYsLMz+vS7JLFy40OE5ERERZvbs2caYnN+/q1atMpJMenq6yczMNJMnTzbh4eEmJSXFpKSkmDNnzhTdF1eM9erVy3Tt2tUYY8zgwYNNbGysWbx4sdm5c6fp1auXKV26tP3v8iuvvGJq167t8PyhQ4eali1bFnXsQnf8+HFTokQJM3HiRJOcnGy2bdtm3nrrLXPmzBnz7bffmvfff9/s2rXL7Nq1y/Tt29dERUWZ06dPG2OMyc7ONnXr1jWtW7c2W7duNWvWrDGNGjVy+B7Oy7+F27ZtM6GhoWbSpElm79695ocffjCNGjUyjz32mDHGmI0bNxp/f38zf/58c+jQIbNlyxbz5ptvOs1fXF0vc69evUx4eLjp37+/2b17t/nyyy9NcHCww79xs2bNMosXLzYHDhww69evN82aNTOdOnWy77/6mdCsWTOzdu1as2XLFhMfH29atWplOnToYLZs2WK+++47U6ZMGfPqq6/an+fscx74o79c0f9naWlpRpLZvn27/YPu3//+t33/zp07jSSze/duY4wxDzzwgOncubPDOR566CGHov/2228348aNczjm/fffNzExMfbHkkxCQkIhfEVF48yZMyYwMNB89NFH9rHffvvNBAUF5Vr0T5gwwVSvXt1cunQpx7l2795tJJmNGzfax/bt22ck5avo79Kli+ndu3eueXN7vq+ZNGmSiYuLM8YYc/bsWVOyZEmzbt06h2P69u1rHnzwwWueY+DAgebee++1P+7Vq5eJiooymZmZhZLZG5w+fdoEBASYTz/91D526tQpExwc7Jai3xhjZs+e7fAZgiuuFv1nz541AQEB5j//+Y9936VLl0xsbKwZP368MeZKUebv729++ukn+/5y5cqZOXPmeCR7Ydq8ebORlKdm0eXLl01YWJj58ssvjTHGLFu2zPj7+5ujR4/aj1myZEmuRf/1/i185JFHzJNPPunwWt9//73x8/MzFy5cMJ999pkJDw+3/7Dhav7i4nqZe/XqZeLi4szly5ftY/fff7954IEHrnm+q42oqz/oXP1M+Oabb+zHJCYmGknmwIED9rF+/fqZjh07GmNc/5zHX9dfbnrPgQMH1LNnT1WpUkXh4eH2aR9HjhyxH1O/fn37n2NiYiTJ/mu4pKQkNWnSxOGcf368efNmvfTSSwoNDbVvTzzxhFJSUhx+Lde4cWP3fnFF6MCBA7p06ZJuvfVW+1hkZKRq1KiR6/H333+/Lly4oCpVquiJJ57QwoUL7dOdkpKSVKJECd1000324+Pj41W6dOl8ZRowYIA++ugjNWzYUCNGjNC6detc+Mp8w65du3Tx4kW1b9/e4ftw3rx5DlMdpk+frsaNG6tcuXIKDQ3VzJkzHf4uSFK9evX+0vP4Dx48qKysLIe/5xEREdf8Xof7HThwQFlZWbrtttvsYwEBAWrSpIl2794t6cpndefOnfXee+9Jkr766itdvHhR999/v0cyF6YGDRqobdu2qlevnu6//37NnDlT6enpkq78W9W/f39Vr15dERERioiI0NmzZ+1/r3fv3q2KFSvqxhtvtJ/vj5/jf3S9fws3b96sOXPmOHy+dOzYUTabTcnJyWrfvr3i4uJUpUoVPfLII/rPf/5j//fvevmLK2eZ69SpI39/f/vjmJgYh+k7W7duVdeuXRUXF6ewsDD7FNM/f97+8T2PiopScHCwqlSp4jB29bx5/ZwHrvrLFf1dunTRb7/9ppkzZ+qnn37STz/9JOnKBYpX/fGCWovFIkmy2WySrsyfuzp2lfnT6hI2m01jxozRzz//bN+2b9+uffv2qWTJkvbjisvKNq7489fsTIUKFZSUlKS33npLQUFBGjhwoFq2bKmsrKxrnuuP435+fjnG/jhfUpI6deqkw4cPKyEhQcePH1fbtm01fPjwfOX0FVe/X7/++muH78Ndu3bZ53t+8sknGjp0qPr06aPly5fr559/Vu/evR3+Lkje/X3qDle/5673995iseT4Pv7z9ydcd73/B38ce/zxx/XRRx/pwoULmj17th544AEFBwcXadai4O/vrxUrVmjJkiWqXbu2pk6dqho1aig5OVmPPfaYNm/erMmTJ2vdunX6+eefVaZMGfvf69w+b//8vl51vX8LbTab+vXr5/D58ssvv2jfvn2qWrWqwsLCtGXLFn344YeKiYnRqFGj1KBBA506deq6+YsrZ5n/vBCHxWKxv1fnzp1Thw4dFBoaqg8++EAbN27UwoULJSnH5+2f3/PrnTcvn/PAH/2liv7ffvtNu3fv1j//+U+1bdtWtWrVynd3oWbNmtqwYYPD2KZNmxwe33TTTUpKSlJ8fHyO7Wrx6u3i4+MVEBCgH3/80T6Wnp6uvXv3XvM5QUFBuvvuuzVlyhStXr1a69ev1/bt21WzZk1dvnxZW7dutR+7f/9+h+ULr64ElJKSYh/L7aLIcuXK6bHHHtMHH3ygyZMna8aMGZJk71RnZ2e79PV6m9q1a8tqterIkSM5vgcrVKggSfr+++/VvHlzDRw4UI0aNVJ8fDzdoVxUrVpVAQEBDn/vT58+rX379tkflytXzuF7c9++fQ6/1XMmMDDwL/O96Yr4+HgFBgZq7dq19rGsrCxt2rRJtWrVso/deeedCgkJ0TvvvKMlS5aoT58+nohbJCwWi2677TaNGTNGW7duVWBgoBYuXKjvv/9egwcP1p133qk6derIarXq5MmT9ufVrl1bR44c0fHjx+1jf75ANy9uuukm7dy5M9d/565+3pYoUULt2rXT+PHjtW3bNh06dEgrV668bv7izNXMe/bs0cmTJ/Xqq6/q9ttvV82aNR1+C+CqvHzOA3/0l1qys3Tp0ipTpoxmzJihmJgYHTlyRM8//3y+zjFo0CC1bNlSEydOVJcuXbRy5UotWbLEoVMyatQo3XXXXapQoYLuv/9++fn5adu2bdq+fbtXrtKTm9DQUPXt21fPPvusypQpo6ioKL3wwgvX/KFmzpw5ys7OVtOmTRUcHKz3339fQUFBiouLs6+C8OSTT+qdd95RQECAhg0bpqCgIPv7GhQUpGbNmunVV19VpUqVdPLkSf3zn/90eI1Ro0bp5ptvVp06dZSZmamvvvrKXhCUL19eQUFBWrp0qW688UaVLFnSp5cDDAsL0/DhwzV06FDZbDa1aNFCp0+f1rp16xQaGqpevXopPj5e8+bN07Jly1S5cmW9//772rhxo33KG64ICwtTr1699OyzzyoyMlLly5fXiy++KD8/P/v35x133KFp06apWbNmstlseu655/K1BG+lSpV09uxZffvtt2rQoIGCg4N9skPtqpCQEA0YMMD+/6BixYoaP368zp8/r759+9qP8/f312OPPaaRI0cqPj7+mtNWvN1PP/2kb7/9Vh06dFD58uX1008/6cSJE6pVq5bi4+P1/vvvq3Hjxjp9+rSeffZZBQUF2Z/brl071ahRQ48++qgmTJig06dP64UXXsh3hueee07NmjXTU089pSeeeEIhISHavXu3VqxYoalTp+qrr77SwYMH1bJlS5UuXVqLFy+WzWZTjRo1rpu/uLpe5m3btl33uRUrVlRgYKCmTp2q/v37a8eOHXr55ZcLnCkvn/OAA89cSuA5K1asMLVq1TJWq9XUr1/frF692n4BU14uFjXGmBkzZpgbbrjBBAUFmXvuuceMHTvWREdHO7zO0qVLTfPmzU1QUJAJDw83TZo0cbiSX7lc+Odtzpw5Yx5++GETHBxsoqKizPjx402rVq1yvZB34cKFpmnTpiY8PNyEhISYZs2aOVywdPz4cdOpUydjtVpNXFycmT9/vilfvryZPn26/Zhdu3aZZs2amaCgINOwYUOzfPlyh/83L7/8sqlVq5YJCgoykZGRpmvXrubgwYP258+cOdNUqFDB+Pn5mVatWhX221Pk/nghrzHG2Gw28+abb5oaNWqYgIAAU65cOdOxY0ezZs0aY4wxFy9eNI899piJiIgwpUqVMgMGDDDPP/+8adCggf0cf1w95a/s9OnTpmfPniY4ONhER0ebiRMnmiZNmpjnn3/eGGPMsWPHTIcOHUxISIipVq2aWbx4cb4u5DXGmP79+5syZcoYSebFF18s2i+wmPrj99+FCxfMoEGDTNmyZY3VajW33Xab2bBhQ47nHDhwwEiyX+Dri3bt2mU6duxoypUrZ6xWq6levbqZOnWqMcaYLVu2mMaNGxur1WqqVatmPv30U4fPYmOMSUpKMi1atDCBgYGmevXqZunSpbleyOvs38INGzaY9u3bm9DQUBMSEmLq169vXnnlFWPMlYt6W7VqZUqXLm2CgoJM/fr17SuDXS9/cXW9zLl9Tg4ZMsTh35n58+ebSpUqGavVam699VbzxRdfOP1MyO0C/xdffNHhM9rZ5zzwRxZjuN1hQT3xxBPas2ePvv/+e09H8Rm//vqrKlSooG+++UZt27b1dBzAwblz53TDDTdowoQJDp1muNeDDz4of39/ffDBB3l+zg8//KDWrVvr119/VVRUVCGmAwDv8pea3uMub7zxhtq3b6+QkBAtWbJEc+fO1dtvv+3pWF5t5cqVOnv2rOrVq6eUlBSNGDFClSpVUsuWLT0dDdDWrVu1Z88eNWnSRBkZGXrppZckSV27dvVwMt90+fJl7d27V+vXr1e/fv3y9JzMzEwdPXpU//rXv9SjRw8KfgD4E9+4qrSIbdiwQe3bt1e9evU0ffp0TZkyRY8//rinY3m1rKws/eMf/1CdOnXUrVs3lStXTqtXr87XvGigML3xxhtq0KCB/S6c33//vcqWLevpWD5px44daty4serUqaP+/fvn6TkffvihatSooYyMDI0fP76QEwKA92F6DwAAAODj6PQDAAAAPo6iHwAAAPBxFP0AAACAj6PoBwAAAHwcRT8AAADg4yj6AQAAAB9H0Q8AAAD4OIp+AAAAwMf9P27pyeXb9DB6AAAAAElFTkSuQmCC",
      "text/plain": [
       "<Figure size 1000x700 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "%matplotlib inline\n",
    "import seaborn as sn\n",
    "import pandas as pd\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "l = [\"joy\", 'fear', \"anger\", \"sadness\", \"disgust\", \"shame\", \"guilt\"]\n",
    "l.sort()\n",
    "df_cm = pd.DataFrame(matrix, index = l, columns = l)\n",
    "plt.figure(figsize = (10,7))\n",
    "sn.heatmap(df_cm, annot=True, fmt=\"d\")\n",
    "plt.show()\n",
    "\n",
    "# Reference https://stackoverflow.com/questions/35572000/how-can-i-plot-a-confusion-matrix"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "joy       (1. 0. 0. 0. 0. 0. 0.)  1084\n",
      "anger     (0. 0. 1. 0. 0. 0. 0.)  1080\n",
      "sadness   (0. 0. 0. 1. 0. 0. 0.)  1079\n",
      "fear      (0. 1. 0. 0. 0. 0. 0.)  1078\n",
      "disgust   (0. 0. 0. 0. 1. 0. 0.)  1057\n",
      "guilt     (0. 0. 0. 0. 0. 0. 1.)  1057\n",
      "shame     (0. 0. 0. 0. 0. 1. 0.)  1045\n"
     ]
    }
   ],
   "source": [
    "# Get counts for each label\n",
    "label_freq = {}\n",
    "for label, _ in psychExp_txt: \n",
    "    label_freq[label] = label_freq.get(label, 0) + 1\n",
    "\n",
    "# print the labels and their counts in sorted order \n",
    "for l in sorted(label_freq, key=label_freq.get, reverse=True):\n",
    "    print(\"{:10}({})  {}\".format(convert_label(l, emotions), l, label_freq[l]))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "By looking at the distributions of the labels, the data is distributed well. The errors probably happen because there are not enough data to train the classifier. Also, ngram with the range of 1 to 4 might not be enough or adding noise to the classifiers."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Future Work \n",
    "\n",
    "To be able to accurately analyze the text, we probably need to have more data to train the classifiers. Also, it would be better to further experiment with features. (I'm not sure what we can use, but probably there is something to make features) At the same time, by adding more data with emoji, we can predict more emoji associated with the texts. \n",
    "\n",
    "<br> \n",
    "For the future work, we want to have more data and more emoji to analyze and represent the sentiment of the texts. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### For fun"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 59,
   "metadata": {
    "tags": []
   },
   "outputs": [],
   "source": [
    "emoji_dict = {\"joy\":\"😂\", \"fear\":\"😱\", \"anger\":\"😠\", \"sadness\":\"😢\", \"disgust\":\"😒\", \"shame\":\"😳\", \"guilt\":\"😳\"}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "😂 :joy: Thank you for dinner!\n",
      "😢 :sadness: I don't like it\n",
      "😱 :fear: My car skidded on the wet street\n",
      "😳 :shame: why are you running\n"
     ]
    }
   ],
   "source": [
    "t1 = \"Thank you for dinner!\"\n",
    "t2 = \"I don't like it\"\n",
    "t3 = \"My car skidded on the wet street\"\n",
    "t4 = \"why are you running\"\n",
    "\n",
    "texts = [t1, t2, t3, t4]\n",
    "for text in texts: \n",
    "    features = create_feature(text, nrange=(1, 4))\n",
    "    features = vectorizer.transform(features)\n",
    "    prediction = grid_obj.predict(features)[0]\n",
    "    print(\"{} :{}: {}\".format(emoji_dict[prediction], prediction, text))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Reference \n",
    "+ [DeepMoji](https://www.media.mit.edu/projects/deepmoji/overview/)\n",
    "+ [DeepMoji GitHub](https://github.com/bfelbo/DeepMoji)\n",
    "+ [Multiclass and multilabel algorithms](http://scikit-learn.org/stable/modules/multiclass.html)\n",
    "+ [sklearn.svm.SVC](http://scikit-learn.org/stable/modules/generated/sklearn.svm.SVC.html)\n",
    "+ [sklearn.svm.LinearSVC](http://scikit-learn.org/stable/modules/generated/sklearn.svm.LinearSVC.html)\n",
    "+ [sklearn.ensemble.RandomForestClassifie](http://scikit-learn.org/stable/modules/generated/sklearn.ensemble.RandomForestClassifier.html)\n",
    "+ [sklearn.tree.DecisionTreeClassifier](http://scikit-learn.org/stable/modules/generated/sklearn.tree.DecisionTreeClassifier.html)\n"
   ]
  },
  {
   "cell_type": "raw",
   "metadata": {
    "jupyter": {
     "outputs_hidden": true
    }
   },
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.7"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}