classes/18/NLP Lesson (Completed).ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Natural Language Processing (NLP)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Introduction\n",
    "\n",
    "*Adapted from [NLP Crash Course](http://files.meetup.com/7616132/DC-NLP-2013-09%20Charlie%20Greenbacker.pdf) by Charlie Greenbacker and [Introduction to NLP](http://spark-public.s3.amazonaws.com/nlp/slides/intro.pdf) by Dan Jurafsky*"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What is NLP?\n",
    "\n",
    "- Using computers to process (analyze, understand, generate) natural human languages\n",
    "- Most knowledge created by humans is unstructured text, and we need a way to make sense of it\n",
    "- Build probabilistic model using data about a language\n",
    "- Also referred to as machine learning with text."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What are some of the higher level task areas?\n",
    "\n",
    "- **Information retrieval**: Find relevant results and similar results\n",
    "    - [Google](https://www.google.com/)\n",
    "- **Information extraction**: Structured information from unstructured documents\n",
    "    - [Events from Gmail](https://support.google.com/calendar/answer/6084018?hl=en)\n",
    "- **Machine translation**: One language to another\n",
    "    - [Google Translate](https://translate.google.com/)\n",
    "- **Text simplification**: Preserve the meaning of text, but simplify the grammar and vocabulary\n",
    "    - [Rewordify](https://rewordify.com/)\n",
    "    - [Simple English Wikipedia](https://simple.wikipedia.org/wiki/Main_Page)\n",
    "- **Predictive text input**: Faster or easier typing\n",
    "    - [A friend's application](https://justmarkham.shinyapps.io/textprediction/)\n",
    "    - [A much better application](https://farsite.shinyapps.io/swiftkey-cap/)\n",
    "- **Sentiment analysis**: Attitude of speaker\n",
    "    - [Hater News](http://haternews.herokuapp.com/)\n",
    "- **Automatic summarization**: Extractive or abstractive summarization\n",
    "    - [autotldr](https://www.reddit.com/r/technology/comments/35brc8/21_million_people_still_use_aol_dialup/cr2zzj0)\n",
    "- **Natural Language Generation**: Generate text from data\n",
    "    - [How a computer describes a sports match](http://www.bbc.com/news/technology-34204052)\n",
    "    - [Publishers withdraw more than 120 gibberish papers](http://www.nature.com/news/publishers-withdraw-more-than-120-gibberish-papers-1.14763)\n",
    "- **Speech recognition and generation**: Speech-to-text, text-to-speech\n",
    "    - [Google's Web Speech API demo](https://www.google.com/intl/en/chrome/demos/speech.html)\n",
    "    - [Vocalware Text-to-Speech demo](https://www.vocalware.com/index/demo)\n",
    "- **Question answering**: Determine the intent of the question, match query with knowledge base, evaluate hypotheses\n",
    "    - [How did supercomputer Watson beat Jeopardy champion Ken Jennings?](http://blog.ted.com/how-did-supercomputer-watson-beat-jeopardy-champion-ken-jennings-experts-discuss/)\n",
    "    - [IBM's Watson Trivia Challenge](http://www.nytimes.com/interactive/2010/06/16/magazine/watson-trivia-game.html)\n",
    "    - [The AI Behind Watson](http://www.aaai.org/Magazine/Watson/watson.php)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### What are some of the lower level components?\n",
    "\n",
    "- **Tokenization**: breaking text into tokens (words, sentences, n-grams)\n",
    "- **Stopword removal**: a/an/the\n",
    "- **Stemming and lemmatization**: root word\n",
    "- **TF-IDF**: word importance\n",
    "- **Part-of-speech tagging**: noun/verb/adjective\n",
    "- **Named entity recognition**: person/organization/location\n",
    "- **Spelling correction**: \"New Yrok City\"\n",
    "- **Word sense disambiguation**: \"buy a mouse\"\n",
    "- **Segmentation**: \"New York City subway\"\n",
    "- **Language detection**: \"translate this page\"\n",
    "- **Machine learning**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Why is NLP hard?\n",
    "\n",
    "- **Ambiguity**:\n",
    "    - Hospitals are Sued by 7 Foot Doctors\n",
    "    - Juvenile Court to Try Shooting Defendant\n",
    "    - Local High School Dropouts Cut in Half\n",
    "- **Non-standard English**: text messages\n",
    "- **Idioms**: \"throw in the towel\"\n",
    "- **Newly coined words**: \"retweet\"\n",
    "- **Tricky entity names**: \"Where is A Bug's Life playing?\"\n",
    "- **World knowledge**: \"Mary and Sue are sisters\", \"Mary and Sue are mothers\"\n",
    "- **Texts with the same words and phrases can having different meanings **: \n",
    "State farm commercial where two different people say \"Is this my car? What? This is ridiculous! This can't be happening! Shut up! Ahhhh!!!\"\n",
    "\n",
    "\n",
    "NLP requires an understanding of the **language** and the **world**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 1: Reading in the Yelp Reviews"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- \"corpus\" = collection of documents\n",
    "- \"corpora\" = plural form of corpus"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "import pandas as pd\n",
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "%matplotlib inline\n",
    "import seaborn as sb\n",
    "from sklearn.cross_validation import train_test_split, cross_val_score\n",
    "from sklearn.feature_extraction.text import CountVectorizer, TfidfVectorizer\n",
    "from sklearn import metrics\n",
    "from textblob import TextBlob, Word\n",
    "from nltk.stem.snowball import SnowballStemmer\n",
    "%matplotlib inline\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "# read yelp.csv into a DataFrame\n",
    "url = 'yelp.csv'\n",
    "yelp = pd.read_csv(url, encoding='unicode-escape')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Create a new DataFrame called yelp_best_worst that only contains the 5-star and 1-star reviews\n",
    "yelp_best_worst = yelp[(yelp.stars==5) | (yelp.stars==1)]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "yelp_best_worst.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "5    0.816691\n",
      "1    0.183309\n",
      "Name: stars, dtype: float64\n"
     ]
    }
   ],
   "source": [
    "# define X and y\n",
    "X = yelp_best_worst.text\n",
    "y = yelp_best_worst.stars\n",
    "print y.value_counts(normalize=True)\n",
    "\n",
    "# split the new DataFrame into training and testing sets\n",
    "X_train, X_test, y_train, y_test = train_test_split(X, y, random_state=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "u'My wife took me here on my birthday for breakfast and it was excellent.  The weather was perfect which made sitting outside overlooking their grounds an absolute pleasure.  Our waitress was excellent and our food arrived quickly on the semi-busy Saturday morning.  It looked like the place fills up pretty quickly so the earlier you get here the better.\\r\\n\\r\\nDo yourself a favor and get their Bloody Mary.  It was phenomenal and simply the best I\\'ve ever had.  I\\'m pretty sure they only use ingredients from their garden and blend them fresh when you order it.  It was amazing.\\r\\n\\r\\nWhile EVERYTHING on the menu looks excellent, I had the white truffle scrambled eggs vegetable skillet and it was tasty and delicious.  It came with 2 pieces of their griddled bread with was amazing and it absolutely made the meal complete.  It was the best \"toast\" I\\'ve ever had.\\r\\n\\r\\nAnyway, I can\\'t wait to go back!'"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X_train[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 2: Tokenization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **What:** Separate text into units such as sentences or words\n",
    "- **Why:** Gives structure to previously unstructured text\n",
    "- **Notes:** Relatively easy with English language text, not easy with some languages"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "From the [scikit-learn documentation](http://scikit-learn.org/stable/modules/feature_extraction.html#text-feature-extraction):\n",
    "\n",
    "> Text Analysis is a major application field for machine learning algorithms. However the raw data, a sequence of symbols cannot be fed directly to the algorithms themselves as most of them expect **numerical feature vectors with a fixed size** rather than the **raw text documents with variable length**.\n",
    "\n",
    "We will use [CountVectorizer](http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html) to \"convert text into a matrix of token counts\":"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# example documents\n",
    "simple_train = ['call you tonight', 'Call me a cab', 'please call me... PLEASE!']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cab</th>\n",
       "      <th>call</th>\n",
       "      <th>me</th>\n",
       "      <th>please</th>\n",
       "      <th>tonight</th>\n",
       "      <th>you</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   cab  call  me  please  tonight  you\n",
       "0    0     1   0       0        1    1\n",
       "1    1     1   1       0        0    0\n",
       "2    0     1   1       2        0    0"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Term Frequency\n",
    "vect = CountVectorizer()\n",
    "dtm = vect.fit_transform(simple_train)\n",
    "tf = pd.DataFrame(dtm.toarray(), columns=vect.get_feature_names())\n",
    "tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cab</th>\n",
       "      <th>call</th>\n",
       "      <th>me</th>\n",
       "      <th>please</th>\n",
       "      <th>tonight</th>\n",
       "      <th>you</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   cab  call  me  please  tonight  you\n",
       "0    1     1   0       1        0    0"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# transforming a new sentence, what do you notice?\n",
    "new_sentence = ['please call yourself a cab burrito']\n",
    "pd.DataFrame(vect.transform(new_sentence).toarray(), columns=vect.get_feature_names())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "# use CountVectorizer to create document-term matrices from X_train and X_test\n",
    "vect = CountVectorizer()\n",
    "X_train_dtm = vect.fit_transform(X_train)\n",
    "X_test_dtm = vect.transform(X_test)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "(3064, 16825)\n",
      "(1022, 16825)\n"
     ]
    }
   ],
   "source": [
    "# rows are documents, columns are terms (phrases) (aka \"tokens\" or \"features\")\n",
    "print X_train_dtm.shape\n",
    "print X_test_dtm.shape\n",
    "# Why do they have the same number of features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[u'00', u'000', u'00a', u'00am', u'00pm', u'01', u'02', u'03', u'03342', u'04', u'05', u'06', u'07', u'09', u'0buxoc0crqjpvkezo3bqog', u'0l', u'10', u'100', u'1000', u'1000x', u'1001', u'100th', u'101', u'102', u'105', u'1070', u'108', u'10am', u'10ish', u'10min', u'10mins', u'10minutes', u'10pm', u'10th', u'10x', u'11', u'110', u'1100', u'111', u'111th', u'112', u'115th', u'118', u'11a', u'11am', u'11p', u'11pm', u'12', u'120', u'128i']\n"
     ]
    }
   ],
   "source": [
    "# first 50 features\n",
    "print vect.get_feature_names()[:50]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[u'jackrabbit', u'jacks', u'jacob', u'jacques', u'jacquie', u'jacuzzi', u'jade', u'jaeger', u'jagermeister', u'jaguar', u'jaime', u'jake', u'jalapeno', u'jalapenos', u'jalape\\xf1o', u'jalepeno', u'jalpeno', u'jam', u'jamaica', u'jamaican', u'jamba', u'jambalaya', u'jamburrito', u'james', u'jameson', u'jammed', u'jamming', u'jamoca', u'jams', u'jan', u'janes', u'janet', u'janis', u'january', u'japan', u'japanese', u'japchae', u'jar', u'jared', u'jargon', u'jars', u'jasmine', u'jason', u'java', u'jaw', u'jaws', u'jay', u'jazz', u'jazzed', u'jc']\n"
     ]
    }
   ],
   "source": [
    "# last 50 features\n",
    "print vect.get_feature_names()[8000:8050]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "CountVectorizer(analyzer=u'word', binary=False, decode_error=u'strict',\n",
       "        dtype=<type 'numpy.int64'>, encoding=u'utf-8', input=u'content',\n",
       "        lowercase=True, max_df=1.0, max_features=None, min_df=1,\n",
       "        ngram_range=(1, 1), preprocessor=None, stop_words=None,\n",
       "        strip_accents=None, token_pattern=u'(?u)\\\\b\\\\w\\\\w+\\\\b',\n",
       "        tokenizer=None, vocabulary=None)"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# show vectorizer options\n",
    "vect"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "[CountVectorizer documentation](http://scikit-learn.org/stable/modules/generated/sklearn.feature_extraction.text.CountVectorizer.html)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **lowercase:** boolean, True by default\n",
    "- Convert all characters to lowercase before tokenizing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3064, 20838)"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Create a count vectorizer that doesn't lowercase the words\n",
    "vect = CountVectorizer(lowercase=False)\n",
    "X_train_dtm = vect.fit_transform(X_train)\n",
    "X_train_dtm.shape # has more features"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **ngram_range:** tuple (min_n, max_n)\n",
    "- The lower and upper boundary of the range of n-values for different n-grams to be extracted. All values of n such that min_n <= n <= max_n will be used."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(3064, 169847)"
      ]
     },
     "execution_count": 18,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# include 1-grams and 2-grams\n",
    "vect = CountVectorizer(ngram_range=(1, 2))\n",
    "X_train_dtm = vect.fit_transform(X_train)\n",
    "X_train_dtm.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# last 50 features\n",
    "print vect.get_feature_names()[-50:]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## **Predicting the star rating with Naive Bayes**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Naive Bayes"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Bayes Theorem covers the probabilistic relationship between multiple variables, and specifically allows us to define one conditional in terms of the underlying probabilities and the inverse condition. Specifically, it can be defined as:\n",
    "\n",
    "$$P(y|x) = P(y)P(x|y)/P(x)$$\n",
    "\n",
    "This means the probability of y given x condition equals the probability of y times the probability of x given y condition divided by the probability of x.\n",
    "\n",
    "This theorem can be extended to when x is a vector (containing the multiple x variables used as inputs for the model) to:\n",
    "\n",
    "$$P(y|x_1,...,x_n) = P(y)P(x_1,...,x_n|y)/P(x_1,...,x_n)$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's pretend we have an email with three words: \"Send money now.\" We'll use Naive Bayes to classify it as **ham or spam.**\n",
    "\n",
    "$$P(spam \\ | \\ \\text{send money now}) = \\frac {P(\\text{send money now} \\ | \\ spam) \\times P(spam)} {P(\\text{send money now})}$$\n",
    "\n",
    "By assuming that the features (the words) are **conditionally independent**, we can simplify the likelihood function:\n",
    "\n",
    "$$P(spam \\ | \\ \\text{send money now}) \\approx \\frac {P(\\text{send} \\ | \\ spam) \\times P(\\text{money} \\ | \\ spam) \\times P(\\text{now} \\ | \\ spam) \\times P(spam)} {P(\\text{send money now})}$$\n",
    "\n",
    "We can calculate all of the values in the numerator by examining a corpus of **spam email**:\n",
    "\n",
    "$$P(spam \\ | \\ \\text{send money now}) \\approx \\frac {0.2 \\times 0.1 \\times 0.1 \\times 0.9} {P(\\text{send money now})} = \\frac {0.0018} {P(\\text{send money now})}$$\n",
    "\n",
    "We would repeat this process with a corpus of **ham email**:\n",
    "\n",
    "$$P(ham \\ | \\ \\text{send money now}) \\approx \\frac {0.05 \\times 0.01 \\times 0.1 \\times 0.1} {P(\\text{send money now})} = \\frac {0.000005} {P(\\text{send money now})}$$\n",
    "\n",
    "All we care about is whether spam or ham has the **higher probability**, and so we predict that the email is **spam**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### Key takeaways\n",
    "\n",
    "- The **\"naive\" assumption** of Naive Bayes (that the features are conditionally independent) is critical to making these calculations simple.\n",
    "- The **normalization constant** (the denominator) can be ignored since it's the same for all classes.\n",
    "- The **prior probability** is much less relevant once you have a lot of features."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### <b>Pros</b>: \n",
    "#### - Very fast. Adept at handling tens of thousands of features which is why it's used for text classification\n",
    "#### - Works well with a small number of observations\n",
    "#### - Isn't negatively affected by \"noise\"\n",
    "\n",
    "### <b>Cons</b>:\n",
    "#### - Useless for probabilities. Most of the time assigns probabilites that are close to zero or one\n",
    "#### - It is literally \"naive\". Meaning it assumes features are independent."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.naive_bayes import MultinomialNB"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.97185511502692123"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#test model on the whole data then do a cross valdiation\n",
    "vect = CountVectorizer()\n",
    "Xdtm = vect.fit_transform(X)\n",
    "nb = MultinomialNB()\n",
    "nb.fit(Xdtm, y)\n",
    "nb.score(Xdtm, y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.918786692759\n"
     ]
    }
   ],
   "source": [
    "# make a countvectorizer for a train test split\n",
    "vect = CountVectorizer()\n",
    "# create document-term matrices\n",
    "X_train_dtm = vect.fit_transform(X_train)\n",
    "X_test_dtm = vect.transform(X_test)\n",
    "\n",
    "# use multinomial naive bayes with document feature matrix, NOT the text column\n",
    "nb = MultinomialNB()\n",
    "nb.fit(X_train_dtm, y_train)\n",
    "y_pred_class = nb.predict(X_test_dtm)\n",
    "# calculate accuracy\n",
    "print metrics.accuracy_score(y_test, y_pred_class)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.81996086105675148"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# calculate null accuracy, which is the accuracy of our null model (just guessing the most common thing)\n",
    "y_test_binary = np.where(y_test==5, 1, 0)\n",
    "max(y_test_binary.mean(), 1 - y_test_binary.mean())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# Predict on new text\n",
    "new_text = [\"I had a decent time at this restaurant. The food was delicious but the service was poor. I recommend the salad but do not eat the french fries.\"]\n",
    "new_text_transform = vect.transform(new_text)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 31,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([5])"
      ]
     },
     "execution_count": 31,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nb.predict(new_text_transform)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 26,
   "metadata": {},
   "outputs": [],
   "source": [
    "# EXERCISE define a function, tokenize_test,  that does five things:\n",
    "def tokenize_test(vect):\n",
    "    nb = MultinomialNB()\n",
    "    X_dtm = vect.fit_transform(X)\n",
    "    print 'Features: ', X_dtm.shape[1]\n",
    "    print 'Accuracy: ', cross_val_score(nb, X_dtm, y, cv=5, scoring='accuracy').mean()\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 30,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Features:  562844\n",
      "Accuracy:  0.786827457459\n"
     ]
    }
   ],
   "source": [
    "# include 1-grams and 2-grams\n",
    "vect = CountVectorizer(ngram_range=(2,3))\n",
    "tokenize_test(vect)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 3: Stopword Removal"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **What:** Remove common words that will likely appear in any text\n",
    "- **Why:** They don't tell you much about your text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# show vectorizer options\n",
    "vect"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **stop_words:** string {'english'}, list, or None (default)\n",
    "- If 'english', a built-in stop word list for English is used.\n",
    "- If a list, that list is assumed to contain stop words, all of which will be removed from the resulting tokens.\n",
    "- If a word is equally like to show up in a rap lyric as medical paper then its most likely a stop word.\n",
    "- Corpus-specific stopwords, that words that aren't regular stopwords but become stopwords depending on the context.\n",
    "- If None, no stop words will be used. max_df can be set to a value in the range [0.7, 1.0) to automatically detect and filter stop words based on intra corpus document frequency of terms."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 32,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Features:  191336\n",
      "Accuracy:  0.917289645057\n"
     ]
    }
   ],
   "source": [
    "# remove English stop words\n",
    "vect = CountVectorizer(stop_words='english', ngram_range=(1, 2))\n",
    "tokenize_test(vect)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "frozenset(['all', 'six', 'less', 'being', 'indeed', 'over', 'move', 'anyway', 'fifty', 'four', 'not', 'own', 'through', 'yourselves', 'go', 'where', 'mill', 'only', 'find', 'before', 'one', 'whose', 'system', 'how', 'somewhere', 'with', 'thick', 'show', 'had', 'enough', 'should', 'to', 'must', 'whom', 'seeming', 'under', 'ours', 'has', 'might', 'thereafter', 'latterly', 'do', 'them', 'his', 'around', 'than', 'get', 'very', 'de', 'none', 'cannot', 'every', 'whether', 'they', 'front', 'during', 'thus', 'now', 'him', 'nor', 'name', 'several', 'hereafter', 'always', 'who', 'cry', 'whither', 'this', 'someone', 'either', 'each', 'become', 'thereupon', 'sometime', 'side', 'two', 'therein', 'twelve', 'because', 'often', 'ten', 'our', 'eg', 'some', 'back', 'up', 'namely', 'towards', 'are', 'further', 'beyond', 'ourselves', 'yet', 'out', 'even', 'will', 'what', 'still', 'for', 'bottom', 'mine', 'since', 'please', 'forty', 'per', 'its', 'everything', 'behind', 'un', 'above', 'between', 'it', 'neither', 'seemed', 'ever', 'across', 'she', 'somehow', 'be', 'we', 'full', 'never', 'sixty', 'however', 'here', 'otherwise', 'were', 'whereupon', 'nowhere', 'although', 'found', 'alone', 're', 'along', 'fifteen', 'by', 'both', 'about', 'last', 'would', 'anything', 'via', 'many', 'could', 'thence', 'put', 'against', 'keep', 'etc', 'amount', 'became', 'ltd', 'hence', 'onto', 'or', 'con', 'among', 'already', 'co', 'afterwards', 'formerly', 'within', 'seems', 'into', 'others', 'while', 'whatever', 'except', 'down', 'hers', 'everyone', 'done', 'least', 'another', 'whoever', 'moreover', 'couldnt', 'throughout', 'anyhow', 'yourself', 'three', 'from', 'her', 'few', 'together', 'top', 'there', 'due', 'been', 'next', 'anyone', 'eleven', 'much', 'call', 'therefore', 'interest', 'then', 'thru', 'themselves', 'hundred', 'was', 'sincere', 'empty', 'more', 'himself', 'elsewhere', 'mostly', 'on', 'fire', 'am', 'becoming', 'hereby', 'amongst', 'else', 'part', 'everywhere', 'too', 'herself', 'former', 'those', 'he', 'me', 'myself', 'made', 'twenty', 'these', 'bill', 'cant', 'us', 'until', 'besides', 'nevertheless', 'below', 'anywhere', 'nine', 'can', 'of', 'your', 'toward', 'my', 'something', 'and', 'whereafter', 'whenever', 'give', 'almost', 'wherever', 'is', 'describe', 'beforehand', 'herein', 'an', 'as', 'itself', 'at', 'have', 'in', 'seem', 'whence', 'ie', 'any', 'fill', 'again', 'hasnt', 'inc', 'thereby', 'thin', 'no', 'perhaps', 'latter', 'meanwhile', 'when', 'detail', 'same', 'wherein', 'beside', 'also', 'that', 'other', 'take', 'which', 'becomes', 'you', 'if', 'nobody', 'see', 'though', 'may', 'after', 'upon', 'most', 'hereupon', 'eight', 'but', 'serious', 'nothing', 'such', 'why', 'a', 'off', 'whereby', 'third', 'i', 'whole', 'noone', 'sometimes', 'well', 'amoungst', 'yours', 'their', 'rather', 'without', 'so', 'five', 'the', 'first', 'whereas', 'once'])\n"
     ]
    }
   ],
   "source": [
    "# set of stop words\n",
    "print vect.get_stop_words()[:30]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 4: Other CountVectorizer Options"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **max_features:** int or None, default=None\n",
    "- If not None, build a vocabulary that only consider the top max_features ordered by term frequency across the corpus."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Features:  500\n",
      "Accuracy:  0.902110055491\n"
     ]
    }
   ],
   "source": [
    "# remove English stop words and only keep 100 features, MUCH FASTER\n",
    "vect = CountVectorizer(stop_words='english', max_features=500)\n",
    "tokenize_test(vect)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[u'10', u'12', u'15', u'20', u'30', u'50', u'able', u'absolutely', u'actually', u'add', u'ago', u'amazing', u'appetizer', u'area', u'arizona', u'arrived', u'ask', u'asked', u'ate', u'atmosphere']\n"
     ]
    }
   ],
   "source": [
    "# all 100 features\n",
    "print vect.get_feature_names()[:20]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Features:  1000\n",
      "Accuracy:  0.900391378043\n"
     ]
    }
   ],
   "source": [
    "# include 1-grams and 2-grams, and limit the number of features\n",
    "vect = CountVectorizer(ngram_range=(1, 2), max_features=1000)\n",
    "tokenize_test(vect)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **min_df:** float in range [0.0, 1.0] or int, default=1\n",
    "- When building the vocabulary ignore terms that have a document frequency strictly lower than the given threshold. This value is also called cut-off in the literature. If float, the parameter represents a proportion of documents, integer absolute counts."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Features:  32700\n",
      "Accuracy:  0.935396625873\n"
     ]
    }
   ],
   "source": [
    "# include 1-grams and 2-grams, and only include terms that appear at least 3 times\n",
    "vect = CountVectorizer(ngram_range=(1, 2), min_df=3)\n",
    "tokenize_test(vect)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 5: Introduction to TextBlob"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "TextBlob: \"Simplified Text Processing\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 39,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "My wife took me here on my birthday for breakfast and it was excellent.  The weather was perfect which made sitting outside overlooking their grounds an absolute pleasure.  Our waitress was excellent and our food arrived quickly on the semi-busy Saturday morning.  It looked like the place fills up pretty quickly so the earlier you get here the better.\r\n",
      "\r\n",
      "Do yourself a favor and get their Bloody Mary.  It was phenomenal and simply the best I've ever had.  I'm pretty sure they only use ingredients from their garden and blend them fresh when you order it.  It was amazing.\r\n",
      "\r\n",
      "While EVERYTHING on the menu looks excellent, I had the white truffle scrambled eggs vegetable skillet and it was tasty and delicious.  It came with 2 pieces of their griddled bread with was amazing and it absolutely made the meal complete.  It was the best \"toast\" I've ever had.\r\n",
      "\r\n",
      "Anyway, I can't wait to go back!\n"
     ]
    }
   ],
   "source": [
    "# print the first review\n",
    "print yelp_best_worst.text[0]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "metadata": {},
   "outputs": [],
   "source": [
    "# save it as a TextBlob object\n",
    "review = TextBlob(yelp_best_worst.text[0])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "WordList([u'My', u'wife', u'took', u'me', u'here', u'on', u'my', u'birthday', u'for', u'breakfast', u'and', u'it', u'was', u'excellent', u'The', u'weather', u'was', u'perfect', u'which', u'made', u'sitting', u'outside', u'overlooking', u'their', u'grounds', u'an', u'absolute', u'pleasure', u'Our', u'waitress', u'was', u'excellent', u'and', u'our', u'food', u'arrived', u'quickly', u'on', u'the', u'semi-busy', u'Saturday', u'morning', u'It', u'looked', u'like', u'the', u'place', u'fills', u'up', u'pretty'])"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# list the words\n",
    "review.words[:50]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[Sentence(\"My wife took me here on my birthday for breakfast and it was excellent.\"),\n",
       " Sentence(\"The weather was perfect which made sitting outside overlooking their grounds an absolute pleasure.\"),\n",
       " Sentence(\"Our waitress was excellent and our food arrived quickly on the semi-busy Saturday morning.\"),\n",
       " Sentence(\"It looked like the place fills up pretty quickly so the earlier you get here the better.\"),\n",
       " Sentence(\"Do yourself a favor and get their Bloody Mary.\")]"
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# list the sentences\n",
    "review.sentences[:5]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [],
   "source": [
    "# some string methods are available\n",
    "# dir(review)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[(u'My', u'PRP$'),\n",
       " (u'wife', u'NN'),\n",
       " (u'took', u'VBD'),\n",
       " (u'me', u'PRP'),\n",
       " (u'here', u'RB'),\n",
       " (u'on', u'IN'),\n",
       " (u'my', u'PRP$'),\n",
       " (u'birthday', u'NN'),\n",
       " (u'for', u'IN'),\n",
       " (u'breakfast', u'NN'),\n",
       " (u'and', u'CC'),\n",
       " (u'it', u'PRP'),\n",
       " (u'was', u'VBD'),\n",
       " (u'excellent', u'JJ'),\n",
       " (u'The', u'DT'),\n",
       " (u'weather', u'NN'),\n",
       " (u'was', u'VBD'),\n",
       " (u'perfect', u'JJ'),\n",
       " (u'which', u'WDT'),\n",
       " (u'made', u'VBD'),\n",
       " (u'sitting', u'VBG'),\n",
       " (u'outside', u'IN'),\n",
       " (u'overlooking', u'VBG'),\n",
       " (u'their', u'PRP$'),\n",
       " (u'grounds', u'NNS'),\n",
       " (u'an', u'DT'),\n",
       " (u'absolute', u'JJ'),\n",
       " (u'pleasure', u'NN'),\n",
       " (u'Our', u'PRP$'),\n",
       " (u'waitress', u'NN'),\n",
       " (u'was', u'VBD'),\n",
       " (u'excellent', u'JJ'),\n",
       " (u'and', u'CC'),\n",
       " (u'our', u'PRP$'),\n",
       " (u'food', u'NN'),\n",
       " (u'arrived', u'VBD'),\n",
       " (u'quickly', u'RB'),\n",
       " (u'on', u'IN'),\n",
       " (u'the', u'DT'),\n",
       " (u'semi-busy', u'JJ'),\n",
       " (u'Saturday', u'NNP'),\n",
       " (u'morning', u'NN'),\n",
       " (u'It', u'PRP'),\n",
       " (u'looked', u'VBD'),\n",
       " (u'like', u'IN'),\n",
       " (u'the', u'DT'),\n",
       " (u'place', u'NN'),\n",
       " (u'fills', u'VBZ'),\n",
       " (u'up', u'RP'),\n",
       " (u'pretty', u'RB'),\n",
       " (u'quickly', u'RB'),\n",
       " (u'so', u'IN'),\n",
       " (u'the', u'DT'),\n",
       " (u'earlier', u'JJR'),\n",
       " (u'you', u'PRP'),\n",
       " (u'get', u'VBP'),\n",
       " (u'here', u'RB'),\n",
       " (u'the', u'DT'),\n",
       " (u'better', u'JJR'),\n",
       " (u'Do', u'VB'),\n",
       " (u'yourself', u'PRP'),\n",
       " (u'a', u'DT'),\n",
       " (u'favor', u'NN'),\n",
       " (u'and', u'CC'),\n",
       " (u'get', u'VB'),\n",
       " (u'their', u'PRP$'),\n",
       " (u'Bloody', u'NNP'),\n",
       " (u'Mary', u'NNP'),\n",
       " (u'It', u'PRP'),\n",
       " (u'was', u'VBD'),\n",
       " (u'phenomenal', u'JJ'),\n",
       " (u'and', u'CC'),\n",
       " (u'simply', u'RB'),\n",
       " (u'the', u'DT'),\n",
       " (u'best', u'JJS'),\n",
       " (u'I', u'PRP'),\n",
       " (u\"'ve\", u'VBP'),\n",
       " (u'ever', u'RB'),\n",
       " (u'had', u'VBD'),\n",
       " (u'I', u'PRP'),\n",
       " (u\"'m\", u'VBP'),\n",
       " (u'pretty', u'JJ'),\n",
       " (u'sure', u'NN'),\n",
       " (u'they', u'PRP'),\n",
       " (u'only', u'RB'),\n",
       " (u'use', u'VBP'),\n",
       " (u'ingredients', u'NNS'),\n",
       " (u'from', u'IN'),\n",
       " (u'their', u'PRP$'),\n",
       " (u'garden', u'NN'),\n",
       " (u'and', u'CC'),\n",
       " (u'blend', u'VB'),\n",
       " (u'them', u'PRP'),\n",
       " (u'fresh', u'JJ'),\n",
       " (u'when', u'WRB'),\n",
       " (u'you', u'PRP'),\n",
       " (u'order', u'NN'),\n",
       " (u'it', u'PRP'),\n",
       " (u'It', u'PRP'),\n",
       " (u'was', u'VBD'),\n",
       " (u'amazing', u'VBG'),\n",
       " (u'While', u'IN'),\n",
       " (u'EVERYTHING', u'NN'),\n",
       " (u'on', u'IN'),\n",
       " (u'the', u'DT'),\n",
       " (u'menu', u'NN'),\n",
       " (u'looks', u'VBZ'),\n",
       " (u'excellent', u'JJ'),\n",
       " (u'I', u'PRP'),\n",
       " (u'had', u'VBD'),\n",
       " (u'the', u'DT'),\n",
       " (u'white', u'JJ'),\n",
       " (u'truffle', u'NN'),\n",
       " (u'scrambled', u'VBD'),\n",
       " (u'eggs', u'NNS'),\n",
       " (u'vegetable', u'JJ'),\n",
       " (u'skillet', u'NN'),\n",
       " (u'and', u'CC'),\n",
       " (u'it', u'PRP'),\n",
       " (u'was', u'VBD'),\n",
       " (u'tasty', u'JJ'),\n",
       " (u'and', u'CC'),\n",
       " (u'delicious', u'JJ'),\n",
       " (u'It', u'PRP'),\n",
       " (u'came', u'VBD'),\n",
       " (u'with', u'IN'),\n",
       " (u'2', u'CD'),\n",
       " (u'pieces', u'NNS'),\n",
       " (u'of', u'IN'),\n",
       " (u'their', u'PRP$'),\n",
       " (u'griddled', u'VBN'),\n",
       " (u'bread', u'NN'),\n",
       " (u'with', u'IN'),\n",
       " (u'was', u'VBD'),\n",
       " (u'amazing', u'VBG'),\n",
       " (u'and', u'CC'),\n",
       " (u'it', u'PRP'),\n",
       " (u'absolutely', u'RB'),\n",
       " (u'made', u'VBD'),\n",
       " (u'the', u'DT'),\n",
       " (u'meal', u'JJ'),\n",
       " (u'complete', u'JJ'),\n",
       " (u'It', u'PRP'),\n",
       " (u'was', u'VBD'),\n",
       " (u'the', u'DT'),\n",
       " (u'best', u'JJS'),\n",
       " (u'toast', u'NN'),\n",
       " (u'I', u'PRP'),\n",
       " (u\"'ve\", u'VBP'),\n",
       " (u'ever', u'RB'),\n",
       " (u'had', u'VBN'),\n",
       " (u'Anyway', u'NNP'),\n",
       " (u'I', u'PRP'),\n",
       " (u'ca', u'MD'),\n",
       " (u\"n't\", u'RB'),\n",
       " (u'wait', u'VB'),\n",
       " (u'to', u'TO'),\n",
       " (u'go', u'VB'),\n",
       " (u'back', u'RB')]"
      ]
     },
     "execution_count": 46,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Parts-of-speech tagging. Identifies nouns, verbs, adverbs, etc...\n",
    "review.tags"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "POS Tags guide: https://www.ling.upenn.edu/courses/Fall_2003/ling001/penn_treebank_pos.html"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 6: Stemming and Lemmatization"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Stemming:**\n",
    "\n",
    "- **What:** Reduce a word to its base/stem/root form\n",
    "- **Why:** Often makes sense to treat related words the same way\n",
    "- **Notes:**\n",
    "    - Uses a \"simple\" and fast rule-based approach\n",
    "    - Stemmed words are usually not shown to users (used for analysis/indexing)\n",
    "    - Some search engines treat words with the same stem as synonyms"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# initialize stemmer\n",
    "stemmer = SnowballStemmer('english')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compare and contrast the words with their stems."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "WordList([u'My', u'wife', u'took', u'me', u'here', u'on', u'my', u'birthday', u'for', u'breakfast', u'and', u'it', u'was', u'excellent', u'The', u'weather', u'was', u'perfect', u'which', u'made', u'sitting', u'outside', u'overlooking', u'their', u'grounds', u'an', u'absolute', u'pleasure', u'Our', u'waitress', u'was', u'excellent', u'and', u'our', u'food', u'arrived', u'quickly', u'on', u'the', u'semi-busy', u'Saturday', u'morning', u'It', u'looked', u'like', u'the', u'place', u'fills', u'up', u'pretty', u'quickly', u'so', u'the', u'earlier', u'you', u'get', u'here', u'the', u'better', u'Do', u'yourself', u'a', u'favor', u'and', u'get', u'their', u'Bloody', u'Mary', u'It', u'was', u'phenomenal', u'and', u'simply', u'the', u'best', u'I', u\"'ve\", u'ever', u'had', u'I', u\"'m\", u'pretty', u'sure', u'they', u'only', u'use', u'ingredients', u'from', u'their', u'garden', u'and', u'blend', u'them', u'fresh', u'when', u'you', u'order', u'it', u'It', u'was'])"
      ]
     },
     "execution_count": 49,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "review.words[:100]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[u'my', u'wife', u'took', u'me', u'here', u'on', u'my', u'birthday', u'for', u'breakfast', u'and', u'it', u'was', u'excel', u'the', u'weather', u'was', u'perfect', u'which', u'made', u'sit', u'outsid', u'overlook', u'their', u'ground', u'an', u'absolut', u'pleasur', u'our', u'waitress', u'was', u'excel', u'and', u'our', u'food', u'arriv', u'quick', u'on', u'the', u'semi-busi', u'saturday', u'morn', u'it', u'look', u'like', u'the', u'place', u'fill', u'up', u'pretti', u'quick', u'so', u'the', u'earlier', u'you', u'get', u'here', u'the', u'better', u'do', u'yourself', u'a', u'favor', u'and', u'get', u'their', u'bloodi', u'mari', u'it', u'was', u'phenomen', u'and', u'simpli', u'the', u'best', u'i', u've', u'ever', u'had', u'i', u\"'m\", u'pretti', u'sure', u'they', u'onli', u'use', u'ingredi', u'from', u'their', u'garden', u'and', u'blend', u'them', u'fresh', u'when', u'you', u'order', u'it', u'it', u'was']\n"
     ]
    }
   ],
   "source": [
    "# stem each word\n",
    "print [stemmer.stem(word) for word in review.words[:100]]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Lemmatization**\n",
    "\n",
    "- **What:** Derive the canonical form ('lemma') of a word\n",
    "- **Why:** Can be better than stemming\n",
    "- **Notes:** Uses a dictionary-based approach (slower than stemming)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from nltk.stem.wordnet import WordNetLemmatizer"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "u'octopi'"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "word = Word('octopi')\n",
    "stemmer.stem(word)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "lem = WordNetLemmatizer()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "u'octopus'"
      ]
     },
     "execution_count": 56,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Try it with words that look very different when pluralized like indices and octopi\n",
    "lem.lemmatize(\"octopi\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Compare and contrast the originals words with their \"lemons\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 61,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[u'amazing', u'While', u'EVERYTHING', u'on', u'the', u'menu', u'looks', u'excellent', u'I', u'had', u'the', u'white', u'truffle', u'scrambled', u'eggs', u'vegetable', u'skillet', u'and', u'it', u'was', u'tasty', u'and', u'delicious', u'It', u'came', u'with', u'2', u'pieces', u'of', u'their', u'griddled', u'bread', u'with', u'was', u'amazing', u'and', u'it', u'absolutely', u'made', u'the', u'meal', u'complete', u'It', u'was', u'the', u'best', u'toast', u'I', u\"'ve\", u'ever', u'had', u'Anyway', u'I', u'ca', u\"n't\", u'wait', u'to', u'go', u'back']\n"
     ]
    }
   ],
   "source": [
    "print [word for word in review.words[100:200]]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 64,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[u'My', u'wife', u'took', u'me', u'here', u'on', u'my', u'birthday', u'for', u'breakfast', u'and', u'it', u'was', u'excellent', u'The', u'weather', u'was', u'perfect', u'which', u'made', u'sitting', u'outside', u'overlooking', u'their', u'grounds', u'an', u'absolute', u'pleasure', u'Our', u'waitress', u'was', u'excellent', u'and', u'our', u'food', u'arrived', u'quickly', u'on', u'the', u'semi-busy', u'Saturday', u'morning', u'It', u'looked', u'like', u'the', u'place', u'fills', u'up', u'pretty', u'quickly', u'so', u'the', u'earlier', u'you', u'get', u'here', u'the', u'better', u'Do', u'yourself', u'a', u'favor', u'and', u'get', u'their', u'Bloody', u'Mary', u'It', u'was', u'phenomenal', u'and', u'simply', u'the', u'best', u'I', u\"'ve\", u'ever', u'had', u'I', u\"'m\", u'pretty', u'sure', u'they', u'only', u'use', u'ingredients', u'from', u'their', u'garden', u'and', u'blend', u'them', u'fresh', u'when', u'you', u'order', u'it', u'It', u'was', u'amazing', u'While', u'EVERYTHING', u'on', u'the', u'menu', u'looks', u'excellent', u'I', u'had', u'the', u'white', u'truffle', u'scrambled', u'eggs', u'vegetable', u'skillet', u'and', u'it', u'was', u'tasty', u'and', u'delicious', u'It', u'came', u'with', u'2', u'pieces', u'of', u'their', u'griddled', u'bread', u'with', u'was', u'amazing', u'and', u'it', u'absolutely', u'made', u'the', u'meal', u'complete', u'It', u'was', u'the', u'best', u'toast', u'I', u\"'ve\", u'ever', u'had', u'Anyway', u'I', u'ca', u\"n't\", u'wait', u'to', u'go', u'back']\n"
     ]
    }
   ],
   "source": [
    "# assume every word is a noun\n",
    "print [word for word in review.words]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 63,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[u'My', u'wife', u'take', u'me', u'here', u'on', u'my', u'birthday', u'for', u'breakfast', u'and', u'it', u'be', u'excellent', u'The', u'weather', u'be', u'perfect', u'which', u'make', u'sit', u'outside', u'overlook', u'their', u'ground', u'an', u'absolute', u'pleasure', u'Our', u'waitress', u'be', u'excellent', u'and', u'our', u'food', u'arrive', u'quickly', u'on', u'the', u'semi-busy', u'Saturday', u'morning', u'It', u'look', u'like', u'the', u'place', u'fill', u'up', u'pretty', u'quickly', u'so', u'the', u'earlier', u'you', u'get', u'here', u'the', u'better', u'Do', u'yourself', u'a', u'favor', u'and', u'get', u'their', u'Bloody', u'Mary', u'It', u'be', u'phenomenal', u'and', u'simply', u'the', u'best', u'I', u\"'ve\", u'ever', u'have', u'I', u\"'m\", u'pretty', u'sure', u'they', u'only', u'use', u'ingredients', u'from', u'their', u'garden', u'and', u'blend', u'them', u'fresh', u'when', u'you', u'order', u'it', u'It', u'be', u'amaze', u'While', u'EVERYTHING', u'on', u'the', u'menu', u'look', u'excellent', u'I', u'have', u'the', u'white', u'truffle', u'scramble', u'egg', u'vegetable', u'skillet', u'and', u'it', u'be', u'tasty', u'and', u'delicious', u'It', u'come', u'with', u'2', u'piece', u'of', u'their', u'griddle', u'bread', u'with', u'be', u'amaze', u'and', u'it', u'absolutely', u'make', u'the', u'meal', u'complete', u'It', u'be', u'the', u'best', u'toast', u'I', u\"'ve\", u'ever', u'have', u'Anyway', u'I', u'ca', u\"n't\", u'wait', u'to', u'go', u'back']\n"
     ]
    }
   ],
   "source": [
    "# assume every word is a verb\n",
    "print [word.lemmatize(pos='v') for word in review.words]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 66,
   "metadata": {},
   "outputs": [],
   "source": [
    "# define a function that accepts text and returns a list of lemmas\n",
    "def word_tokenize_stem(text):\n",
    "    words = TextBlob(text).words\n",
    "    return [stemmer.stem(word) for word in words]\n",
    "def word_tokenize_lemma(text):\n",
    "    words = TextBlob(text).words\n",
    "    return [word.lemmatize() for word in words]\n",
    "def word_tokenize_lemma_verb(text):\n",
    "    words = TextBlob(text).words\n",
    "    return [word.lemmatize(pos=\"v\") for word in words]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 67,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Features:  15237\n",
      "Accuracy:  0.920713524267\n"
     ]
    }
   ],
   "source": [
    "# use word_tokenize LEMMA as the feature extraction function (WARNING: SLOW!)\n",
    "# this will lemmatize each word\n",
    "vect = CountVectorizer(analyzer=word_tokenize_stem)\n",
    "tokenize_test(vect)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 68,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Features:  23856\n",
      "Accuracy:  0.912390384251\n"
     ]
    }
   ],
   "source": [
    "# use word_tokenize STEM as the feature extraction function (WARNING: SLOW!)\n",
    "# this will lemmatize each word\n",
    "vect = CountVectorizer(analyzer=word_tokenize_lemma)\n",
    "tokenize_test(vect)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 7: Term Frequency-Inverse Document Frequency (TF-IDF)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- **What:** Computes \"relative frequency\" that a word appears in a document compared to its frequency across all documents\n",
    "- **Why:** More useful than \"term frequency\" for identifying \"important\" words in each document (high frequency in that document, low frequency in other documents). Court, ball, shooting, passing will show up frequently in a basketball corpus, but essentially add no meaning.\n",
    "- **Notes:** Used for search engine scoring, text summarization, document clustering"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 69,
   "metadata": {},
   "outputs": [],
   "source": [
    "# example documents\n",
    "simple_train = ['call you tonight', 'Call me a cab', 'please call me... PLEASE!']"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 70,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cab</th>\n",
       "      <th>call</th>\n",
       "      <th>me</th>\n",
       "      <th>please</th>\n",
       "      <th>tonight</th>\n",
       "      <th>you</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>2</td>\n",
       "      <td>0</td>\n",
       "      <td>0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   cab  call  me  please  tonight  you\n",
       "0    0     1   0       0        1    1\n",
       "1    1     1   1       0        0    0\n",
       "2    0     1   1       2        0    0"
      ]
     },
     "execution_count": 70,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Term Frequency\n",
    "vect = CountVectorizer()\n",
    "tf = pd.DataFrame(vect.fit_transform(simple_train).toarray(), columns=vect.get_feature_names())\n",
    "tf"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 71,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cab</th>\n",
       "      <th>call</th>\n",
       "      <th>me</th>\n",
       "      <th>please</th>\n",
       "      <th>tonight</th>\n",
       "      <th>you</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>1</td>\n",
       "      <td>3</td>\n",
       "      <td>2</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "      <td>1</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   cab  call  me  please  tonight  you\n",
       "0    1     3   2       1        1    1"
      ]
     },
     "execution_count": 71,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Document Frequency\n",
    "vect = CountVectorizer(binary=True)\n",
    "df = vect.fit_transform(simple_train).toarray().sum(axis=0)\n",
    "pd.DataFrame(df.reshape(1, 6), columns=vect.get_feature_names())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 72,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cab</th>\n",
       "      <th>call</th>\n",
       "      <th>me</th>\n",
       "      <th>please</th>\n",
       "      <th>tonight</th>\n",
       "      <th>you</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.333333</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.333333</td>\n",
       "      <td>0.5</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.0</td>\n",
       "      <td>0.333333</td>\n",
       "      <td>0.5</td>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "   cab      call   me  please  tonight  you\n",
       "0  0.0  0.333333  0.0     0.0      1.0  1.0\n",
       "1  1.0  0.333333  0.5     0.0      0.0  0.0\n",
       "2  0.0  0.333333  0.5     2.0      0.0  0.0"
      ]
     },
     "execution_count": 72,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Term Frequency-Inverse Document Frequency (simple version)\n",
    "tf/df"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 73,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>cab</th>\n",
       "      <th>call</th>\n",
       "      <th>me</th>\n",
       "      <th>please</th>\n",
       "      <th>tonight</th>\n",
       "      <th>you</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.385372</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.652491</td>\n",
       "      <td>0.652491</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>0.720333</td>\n",
       "      <td>0.425441</td>\n",
       "      <td>0.547832</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.266075</td>\n",
       "      <td>0.342620</td>\n",
       "      <td>0.901008</td>\n",
       "      <td>0.000000</td>\n",
       "      <td>0.000000</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "        cab      call        me    please   tonight       you\n",
       "0  0.000000  0.385372  0.000000  0.000000  0.652491  0.652491\n",
       "1  0.720333  0.425441  0.547832  0.000000  0.000000  0.000000\n",
       "2  0.000000  0.266075  0.342620  0.901008  0.000000  0.000000"
      ]
     },
     "execution_count": 73,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# TfidfVectorizer. Why does please have the highest score?\n",
    "vect = TfidfVectorizer()\n",
    "pd.DataFrame(vect.fit_transform(simple_train).toarray(), columns=vect.get_feature_names())"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**More details:** [TF-IDF is about what matters](http://planspace.org/20150524-tfidf_is_about_what_matters/)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 74,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(10000, 28880)"
      ]
     },
     "execution_count": 74,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create a document-term matrix using TF-IDF\n",
    "vect = TfidfVectorizer(stop_words='english')\n",
    "dtm = vect.fit_transform(yelp.text)\n",
    "features = vect.get_feature_names()\n",
    "dtm.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 75,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Features:  18883\n",
      "Accuracy:  0.817671138582\n"
     ]
    }
   ],
   "source": [
    "vect = TfidfVectorizer(stop_words='english')\n",
    "tokenize_test(vect)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 8: Sentiment Analysis"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 76,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "My wife took me here on my birthday for breakfast and it was excellent.  The weather was perfect which made sitting outside overlooking their grounds an absolute pleasure.  Our waitress was excellent and our food arrived quickly on the semi-busy Saturday morning.  It looked like the place fills up pretty quickly so the earlier you get here the better.\r\n",
      "\r\n",
      "Do yourself a favor and get their Bloody Mary.  It was phenomenal and simply the best I've ever had.  I'm pretty sure they only use ingredients from their garden and blend them fresh when you order it.  It was amazing.\r\n",
      "\r\n",
      "While EVERYTHING on the menu looks excellent, I had the white truffle scrambled eggs vegetable skillet and it was tasty and delicious.  It came with 2 pieces of their griddled bread with was amazing and it absolutely made the meal complete.  It was the best \"toast\" I've ever had.\r\n",
      "\r\n",
      "Anyway, I can't wait to go back!\n"
     ]
    }
   ],
   "source": [
    "print review"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 77,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Sentiment(polarity=0.40246913580246907, subjectivity=0.6591122868900646)"
      ]
     },
     "execution_count": 77,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "review.sentiment"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 78,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Apply polarity and sentiment over yelp reviews df\n",
    "yelp[\"polarity\"] = yelp.text.apply(lambda x:TextBlob(x).polarity)\n",
    "yelp[\"subjectivity\"] = yelp.text.apply(lambda x:TextBlob(x).subjectivity)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 79,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "yelp[\"review_length\"] = yelp.text.str.len()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 80,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pd.set_option('max_colwidth', 500)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 81,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "254                                                                                                                                                                             Our server Gary was awesome. Food was amazing...an experience.\n",
       "347                                                                                                                                                           3 syllables for this place. \\r\\nA-MAZ-ING!\\r\\n\\r\\nThe best Phoenix has to offer.\n",
       "420                                                                                                                                                                                                                          LOVE the food!!!!\n",
       "459    Love it!!! Wish we still lived in Arizona as Chino is the one thing we miss. Every time I think about Chino Bandido my mouth starts watering. If I am ever in the state again I will drive out of my way just to go to it again. YUMMY!\n",
       "679                                                                                                                                                                                                                           Excellent burger\n",
       "Name: text, dtype: object"
      ]
     },
     "execution_count": 81,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "yelp[yelp.polarity == 1].text.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 82,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "773     This was absolutely horrible. I got the supreme pizza with the mystery meats.  I threw it in the trash. I will wait until I get to my destination to eat. Horrible!!!\n",
       "1517                                                                                                                                      Nasty workers and over priced trash\n",
       "3266                                                                                                         Absolutely awful... these guys have NO idea what they are doing!\n",
       "4766                                                                                                                                                           Very bad food!\n",
       "5812                                                                                                                            I wouldn't send my worst enemy to this place.\n",
       "Name: text, dtype: object"
      ]
     },
     "execution_count": 82,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "yelp[yelp.polarity == -1].text.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 83,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "390     RIP AZ Coffee Connection.  :(  I stopped by two days ago unaware that they had closed.  I am severely bummed.  This place is irreplaceable!  Damn you, Starbucks and McDonalds!\n",
       "1287                                             Obsessed. Like, I've-got-the-Twangy-Tart-withdrawal-shakes level of addiction to this place. Please make one in Arcadia! Pleeeaaassse.\n",
       "Name: text, dtype: object"
      ]
     },
     "execution_count": 83,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "yelp[(yelp.stars == 5) & (yelp.polarity < -0.3)][\"text\"].head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 84,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1781                                                                                                                                                                                                                                                                     If you like the stuck up Scottsdale vibe this is a good place for you. The food isn't impressive. Nice outdoor seating.\n",
       "2353    My co-workers and I refer to this place as \"Pizza n' Ants\".  The staff will be happy to serve you with bare hands, right after using the till.  Also, as the nickname suggests, there has been a noticable insect problem. \\r\\r\\n\\r\\r\\nAs if that could all be overlooked, the pizza isn't even good.  If you are in this part of town, go to Z Pizza or Slices for great pizza instead!\n",
       "Name: text, dtype: object"
      ]
     },
     "execution_count": 84,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "yelp[(yelp.stars == 1) & (yelp.polarity > 0.5)][\"text\"].head(2)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 85,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAECCAYAAADQEYGEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFSlJREFUeJzt3XuQXOV55/HvWCCDrNEYOSPWC45VKPBAudbYIsGxl+Xi\nGF9I4ksqNrvYXoI3KMUCBSSmsmCT1LpWaycEJyjxUhsuNja5YOw1i024uBZnLRmHALHXq0R+JEdI\nWldhJDRiNGMJJI1m/zgtaAahOfN293S35vupUqn77dOnn3nrTP/mnPec9wxMTk4iSdJMvaLbBUiS\n+pMBIkkqYoBIkooYIJKkIgaIJKmIASJJKnJEp1YcEUcAtwFLgfnASuD/Ad8A1jcWuykz74qIi4EV\nwF5gZWbeGxFHAXcAS4CdwIWZub1T9UqSZmagU9eBRMRvAG/MzN+OiGOA7wP/GRjKzD9uWu5Y4JvA\ncmABsAY4DbgMGMzMT0XE+cBbM/PKjhQrSZqxju2BAF8G7mo8fgXV3sVpwMkR8X6qvZCrgNOBNZm5\nD9gZERuAU4EzgD9ovP8+4LoO1ipJmqGOjYFk5q7M/GlEDFIFySeBvwc+nplnARuB3wcWAaNNbx0H\nhoDBpvaxxnKSpB7R0UH0iHgd8BBwe2b+NXB3Zn6v8fLdwJuoQqI5HAaBHVTjHoNNbc90slZJ0sx0\nchD9WOAB4NLM/Faj+YGIuCwzHwN+CXgceBRYGRHzgaOBk4G1wMPAecBjjf9X1/ncycnJyYGBgbb+\nLJI0B8z4i7OTg+h/AnwI+CFVYZPAJ4DrgT3AT4AVmTkeEf8B+K3Gcisz8+6IOBq4HXgt8BxwQWZu\nrfHRk9u2jbX955mLhocHsS/bx/5sL/uzvYaHB3snQLrIAGkTf0Hby/5sL/uzvUoCxAsJJUlFDBBJ\nUhEDRJJUxACRJBUxQCRJRQwQSVIRA0SSVMQAkSQVMUAkSUUMEElSEQNEklTEAJEkFTFAJElFOnlL\nW0k9ZmJigk2bNra0jqVLT2DevHltqkj9zACR5pBNmzZyxfX3sGBoSdH7d41u5car38uyZSe2uTL1\nIwNEmmMWDC1h4THHdbsMHQYcA5EkFTFAJElFDBBJUhEDRJJUxACRJBUxQCRJRQwQSVIRA0SSVMQA\nkSQVMUAkSUUMEElSEQNEklTEAJEkFTFAJElFDBBJUhEDRJJUxACRJBXxjoRSn2jH/cy3bNncpmok\nA0TqG63ezxxg+4/X8ZrjT2ljVZrLDBCpj7R6P/Ndo0+1sRrNdY6BSJKKdGwPJCKOAG4DlgLzgZXA\nPwFfAPYDazPz0sayFwMrgL3Aysy8NyKOAu4AlgA7gQszc3un6pUkzUwn90A+AjydmWcC7wb+DPgs\ncG1mngW8IiLeFxHHApcDb20s9+mIOBK4BPhB4/1fAq7rYK2SpBnqZIB8mRe+9OcB+4Dlmbm60XYf\ncC5wOrAmM/dl5k5gA3AqcAZwf9Oy7+hgrZKkGerYIazM3AUQEYPAXcAngD9qWmQMWAQMAqNN7ePA\n0JT2A8tKknpER8/CiojXAf8D+LPM/OuI+MOmlweBZ6jGNxZNad/RaB+csmwtw8OD0y+kWuzL9mql\nP3fsWNjGSsotXrywZ7aLXqljrurkIPqxwAPApZn5rUbz9yLizMz8NvAe4CHgUWBlRMwHjgZOBtYC\nDwPnAY81/l9NTdu2jbXt55jLhocH7cs2arU/R0bG21hNuZGR8Z7YLtw+26skjDu5B3IN8Grguoj4\nPWASuAL408Yg+TrgK5k5GRGrgDXAANUg+56IuAm4PSJWA88BF3SwVknSDHVyDORK4MqDvHT2QZa9\nFbh1Sttu4EMdKU6S1DIvJJQkFTFAJElFDBBJUhEDRJJUxACRJBUxQCRJRQwQSVIRA0SSVMQAkSQV\nMUAkSUUMEElSEQNEklTEAJEkFTFAJElFDBBJUhEDRJJUxACRJBUxQCRJRQwQSVIRA0SSVMQAkSQV\nMUAkSUWO6HYBkvrH5P79bNmyueX1LF16AvPmzWtDReomA0RSbbvHtnHDnU+zYOjJ4nXsGt3KjVe/\nl2XLTmxjZeoGA0TSjCwYWsLCY47rdhnqAY6BSJKKGCCSpCIGiCSpiAEiSSpigEiSihggkqQiBogk\nqYgBIkkqYoBIkooYIJKkIgaIJKlIx+fCioi3AJ/JzHMi4k3AN4D1jZdvysy7IuJiYAWwF1iZmfdG\nxFHAHcASYCdwYWZu73S9kqR6OhogEXE18FFgvNF0GnBDZv5x0zLHApcDy4EFwJqIeBC4BPhBZn4q\nIs4HrgOu7GS9kqT6Or0H8iPgA8CXGs9PA06KiPdT7YVcBZwOrMnMfcDOiNgAnAqcAfxB4333UQWI\nJKlH1BoDiYi/iYgPRsSRM1l5Zn4N2NfU9AhwdWaeBWwEfh9YBIw2LTMODAGDTe1jjeUkST2i7iD6\nZ4B3Axsi4nMR8QuFn3d3Zn7vwGPgTVQh0RwOg8AOqnGPwaa2Zwo/U5LUAbUOYWXmt4FvR8TRwK8D\nX42IncAtVAPhz9X8vAci4rLMfAz4JeBx4FFgZUTMB44GTgbWAg8D5wGPNf5fXfeHGh4enH4h1WJf\ntlcr/bljx8I2VtJdixcvbMu25fbZXbXHQCLibKoB8XdSjUncCZwL3AO8q+ZqLgH+NCL2AD8BVmTm\neESsAtYAA8C1mbknIm4Cbo+I1cBzwAV1a922bazuojqE4eFB+7KNWu3PkZHx6RfqEyMj4y1vW26f\n7VUSxrUCJCI2U41ZfB64LDN3N9r/lmoP4mVl5mbgbY3H36MaHJ+6zK3ArVPadgMfqlOfJGn21R0D\neTtwfmZ+ESAifg4gMycyc3mnipMk9a66AfLLwP2Nx0uAr0fEis6UJEnqB3UDZAXwb+D5Q1KnUV38\nJ0mao+oGyJFUA9kH7AEm21+OJKlf1D0L627goYj4cuP5r1GdfSVJmqNq7YFk5u8Cq4AATgBWZeYn\nO1mYJKm3zWQ693XAl6n2RkYi4szOlCRJ6gd1rwP5HPCrwD83NU9Snd4rSZqD6o6BvBOIAxcQSpJU\n9xDWRqppRiRJAurvgYwA/xQRDwPPHmjMzI91pCpJUs+rGyD388KV6JIk1Z7O/faIWAq8AXgAeF1m\nPtHJwiRJva3uHQnPB74O3AgsBr4bER/pZGGSpN5WdxD9d6mmZB/LzK3Am4FrOlaVJKnn1Q2Qicx8\n/s4tmfkksL8zJUmS+kHdQfR/jIjLgCMj4k3AfwS+37mypMPLxMQE69evb+muglu2bG5jRVLr6gbI\npcAngd3AbcBDwO90qijpcLNp00auuP4eFgwtKV7H9h+v4zXHn9LGqqTW1D0L66dUYx6Oe0iFFgwt\nYeExxxW/f9foU22sRmpd3bmw9vPS+388mZnHt78kSVI/qLsH8vxge0QcCbwfeGunipIk9b6ZTOcO\nQGbuzcy7cCZeSZrT6h7C+vdNTweorkjf05GKJEl9oe5ZWOc0PZ4EngbOb385kqR+UXcM5KJOFyJJ\n6i91D2E9wUvPwoLqcNZkZp7Q1qokST2v7iGsvwSeA24G9gIfBn4B+ESH6pIk9bi6AfKuzPz5puc3\nRsTjmencCpI0R9U9jXcgIt5x4ElE/AqwszMlSZL6Qd09kBXAFyPiX1CNhfwQuLBjVUmSel7ds7Ae\nB94QET8DPJuZ5VOKSpIOC3XvSPj6iPgm8F1gYUQ81LjFrSRpjqo7BvLfgeuBceAp4K+AL3aqKElS\n76sbID+TmQ8CZOZkZt4MLOpcWZKkXlc3QHZHxPE0LiaMiDOorguRJM1Rdc/Cugr4BrAsIr4PLAY+\n2LGqJEk9r26AHEt15flJwDzgh5npbLySNIfVDZA/zMx7gX+c6QdExFuAz2TmORGxDPgCsB9Ym5mX\nNpa5mOpak73Aysy8NyKOAu4AllBdtHhhZm6f6edLkjqjboD8c0TcBjwC7D7QmJmHPBMrIq4GPkp1\n9hbAZ4FrM3N1RNwUEe8D/g64HFgOLADWRMSDwCXADzLzUxFxPnAdcGX9H02S1EmHHESPiOMaD7dT\nzbz7i1T3BjkHOLvG+n8EfKDp+WmZubrx+D7gXOB0YE1m7svMncAG4FTgDOD+pmXfgSSpZ0y3B/J1\nYHlmXhQRv5OZN8xk5Zn5tYh4fVPTQNPjMapTgQeB0ab2cWBoSvuBZSVJPWK603ibv/A/3IbP29/0\neBB4hmp8Y9GU9h2N9sEpy0qSesR0eyDNN5EaeNml6vuHiDgzM78NvAd4CHgUWBkR84GjgZOBtcDD\nwHnAY43/Vx98lS81PDw4/UKqxb5sjx07Fna7hJ6yePHCtmxbbp/dVXcQHQ5+R8KZ+jhwc0QcCawD\nvpKZkxGxClhDFVLXZuaeiLgJuD0iVlNdtHhB3Q/Ztm2sDaVqeHjQvmyTkRHnH202MjLe8rbl9tle\nJWE8XYC8ISI2Nh4f1/S49q1sGzedelvj8QYOMviembcCt05p2w18aLr1S5K6Y7oAOWlWqpAk9Z1D\nBoi3rJUkvZy6kylKkvQiBogkqYgBIkkqMpPTeCWpZZP797NlS+vDq4sXn9qGatQKA0TSrNo9to0b\n7nyaBUNPFq9j1+hWvvTphRxzzGvbWJlmygCRNOsWDC1h4THHTb+geppjIJKkIgaIJKmIASJJKmKA\nSJKKGCCSpCIGiCSpiAEiSSpigEiSihggkqQiBogkqYgBIkkqYoBIkooYIJKkIgaIJKmIASJJKmKA\nSJKKGCCSpCIGiCSpiAEiSSpigEiSihggkqQiBogkqYgBIkkqYoBIkooYIJKkIgaIJKmIASJJKmKA\nSJKKGCCSpCJHdONDI+JxYLTx9AngvwJfAPYDazPz0sZyFwMrgL3Aysy8d/arlSQdzKwHSES8EiAz\n397U9j+BazNzdUTcFBHvA/4OuBxYDiwA1kTEg5m5d7ZrliS9VDf2QE4FXhURDwDzgE8AyzNzdeP1\n+4B3Uu2NrMnMfcDOiNgAvBF4vAs1S5Km6MYYyC7g+sx8F3AJ8BfAQNPrY8AiYJAXDnMBjANDs1Wk\nJOnQurEHsh74EUBmboiI7VSHqQ4YBJ4BdlIFydT2aQ0PD7anUtmXbbJjx8Jul3BYcvvsrm4EyMeA\nfwVcGhH/kiokHoyIszLzfwPvAR4CHgVWRsR84GjgZGBtnQ/Ytm2sI4XPNcPDg/Zlm4yMjHe7hMPK\n5P79PPHEEy3369KlJzBv3rw2VdXfSsK4GwFyK/D5iFhNNc7xG8B24JaIOBJYB3wlMycjYhWwhuoQ\n17WZuacL9UrqMbvHtvF7f/40C4aWFK9j1+hWbrz6vSxbdmIbK5tbZj1AGmdRfeQgL519kGVvpQoc\nSXqRBUNLWHjMcd0uY07rynUgUj+ZmJhg06aNLa1jy5bNbapG6h0GiDSNTZs2csX197R0uGT7j9fx\nmuNPaWNVUvcZIFINrR4u2TX6VBurkXqDc2FJkooYIJKkIgaIJKmIASJJKmKASJKKGCCSpCIGiCSp\niAEiSSpigEiSihggkqQiBogkqYgBIkkqYoBIkooYIJKkIgaIJKmI9wORNCdN7t/f8p0ily49gXnz\n5rWpov5jgEiak3aPbeOGO59mwdCTRe/fNbqVG69+L8uWndjmyvqHASJpzmr1TpNznWMgkqQiBogk\nqYiHsHTYm5iYYNOmjcXvb3WgVTpcGSA67G3atJErrr+HBUNLit6//cfreM3xp7S5Kqn/GSCaE1oZ\nLN01+lSbq5EOD46BSJKKGCCSpCIGiCSpiAEiSSpigEiSihggkqQiBogkqYjXgaintXoVOXgluTqj\nHdPBQ+tTwrfjdwRgeHj5jN9jgKintXoVOXgluTqj1engAX76zE/4+L99Mz/7s68vXseWLZu54c7/\n09LvyK7RrTzyVQNEh6FWp9z2SnJ1Sju2zerLvzyEDvyB1I1p6Xs6QCJiAPhvwKnAs8BvZmbr+2qa\nFR5+kqbXz38g9XSAAO8HXpmZb4uItwCfbbSpD3j4STq89XqAnAHcD5CZj0TEz3e5njljYmKC9evX\nMzIyXryOLVs29/VfV5IOrdcDZBEw2vR8X0S8IjP3d6ug6bTrjIhun5nRjoE59x6kw1uvB8hOYLDp\n+bThcfa7P8i+fRPFHxjLlnLxRR8pfv+WLZv5Lzd/k6MWLi5ex7PjI3zy4nNbPjOjlTpGn9rIq197\nUvHnH7BrdGtL7989NgIMdHUdvVBDr6yjF2rolXX0Qg3tWkfp7+nA5ORkSx/cSRHxa8CvZObHIuIX\ngesy85e7XZckqff3QL4GnBsR32k8v6ibxUiSXtDTeyCSpN7lXFiSpCIGiCSpiAEiSSpigEiSivT6\nWVi1RMQHgF/PzA8f5LWLgRXAXmBlZt472/X1i4g4CrgDWEJ1Dc6Fmbl9yjJ/AvxrYKzR9L7MHEPA\n9PO3RcSvAtdRbY+fz8xbulJon6jRn1cCvwkcuJDhtzJzw6wX2mcaU0N9JjPPmdI+o+2z7/dAGl9o\nKznIlTQRcSxwOfBW4N3ApyPiyNmtsK9cAvwgM88EvkS1IU11GvCuzHx745/h8WLPz98GXEM1fxsA\nEXFE4/k7gLOBFREx3I0i+8jL9mfDacBHm7ZHw2MaEXE1cDPwyintM94++z5AgO9QffEdzOnAmszc\nl5k7gQ3AG2etsv7z/NxjwH1UG9LzGn8Nngj8eUSsiQivy3mpF83fBjTP33YKsCEzd2bmXmANcObs\nl9hXDtWfUAXINRGxOiL+02wX16d+BHzgIO0z3j775hBWRHwMuAqYpNrbmAQuysy7IuKsl3nb1Lm0\nxoGhjhbaJ6b0J1R9+hNe6K8xqv5r9ipgFdVfKUcA34qIRzNzbecr7huHmr9t6mtjuD1OZ7r58P4K\n+BzVIde7I+K8zPyb2S6yn2Tm1yLiYPMkzXj77JsAyczbgNtm+LadvPhLcBB4pm1F9bGD9WdEfJUX\n5h47WF/tAlZl5rON5R+iOjZtgLzgUPO3uT3O3HTz4d3YOLpARNwLvBkwQMrMePs8HA5hHcrfA2dE\nxPyIGAJOxi+7Q/kOcF7j8XnA6imvnwR8JyIGGmNJZwD/MIv19YPn+7Axf9v/bXptHfBzEfHqiJhP\ndXjgu7NfYl952f6MiEXA2ohY0Di8+nbg8a5U2Z+mjhvPePvsmz2QmYiIq6iO5X0jIlZRHcsbAK7N\nzD3dra6n3QTcHhGrgeeAC+Al/flF4BFgD3B7Zq7rWrW96SXzt0XEvwNelZm3RMRvAw9SbY+3ZGb5\nvUznhun68xrgb6nO0PpfmXn/y6xHLzUJ0Mr26VxYkqQih/shLElShxggkqQiBogkqYgBIkkqYoBI\nkooYIJKkIgaIJKmIASJJKvL/ASF7bim73kXDAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x118b21d10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "yelp.polarity.plot(kind=\"hist\", bins=20);"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 86,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x112043650>"
      ]
     },
     "execution_count": 86,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZAAAAECCAYAAADQEYGEAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAE/tJREFUeJzt3XmQXWWZx/Fv2ywS08QEO6hgmSLCA+WCA6JCIYv7isuM\nUiqKOhILkcFlKEcctMaqKA6DM6AOI4Ibjg4uA6LIMhajJuJYyLih+CSQrahCEtKh05lESLp7/rg3\ndtsk3affvmv39/NP9z197rlPv3X7/vo973nf0zM6OookSdP1qHYXIEnqTgaIJKmIASJJKmKASJKK\nGCCSpCIGiCSpyD7NOnBE7AN8AVgC7AcsB34HfAkYAe7MzHPq+54FLAN2Assz84aIeDTwVWAxsBU4\nMzM3N6teSdL0NLMHcgbwQGaeBLwU+AzwKeCCzDwZeFREvDoiDgbOBY6v7/eJiNgXOBv4df35VwMX\nNrFWSdI0NTNAvsHYh34vsAs4JjNX1LfdCLwIeDawMjN3ZeZWYDVwNHAicNO4fV/YxFolSdPUtFNY\nmbkdICL6gG8CHwb+adwuQ8CBQB8wOG77NmDBhO2795UkdYimDqJHxJOAW4EvZ+Z/UBv72K0PeJDa\n+MaBE7ZvqW/vm7CvJKlDNHMQ/WDgZuCczPzv+uZfRMRJmflj4GXUwuV2YHlE7AccABwJ3AncBrwc\n+Hn96woqGB0dHe3p6Wno7yJJc8C0Pzh7mrWYYkT8C/AG4PfUChsFzgM+DewL3AWclZmjEfHXwLvq\n+y3PzOsi4gDgy8ATgIeAN2XmxgovPbpp01DDf59u1N/fh21RY1uMsS3G2BZj+vv7OidA2sgAqfOP\nY4xtMca2GGNbjCkJECcSSpKKGCCSpCIGiCSpiAEiSSpigEiSihggkqQiBogkqYgBIkkqYoBIkoo0\nbS0sSY01PDzMunVrZnycRYuObkA1kgEidY1169Zw3sXXM2/B4uJjbB/cyNWfmM/ChU9oYGWaqwwQ\nqYvMW7CY+QsPaXcZEuAYiCSpkAEiSSpigEiSihggkqQiBogkqYhXYUkt0Ig5HBs2rG9QNVJjGCBS\nCzRiDsfme+/ioEOPamBV0swYIFKLzHQOx/bB+xtYjTRzjoFIkooYIJKkIgaIJKmIASJJKmKASJKK\nGCCSpCIGiCSpiAEiSSpigEiSihggkqQiBogkqYgBIkkqYoBIkooYIJKkIgaIJKmIASJJKmKASJKK\nGCCSpCIGiCSpiAEiSSpigEiSihggkqQiBogkqYgBIkkqYoBIkooYIJKkIgaIJKmIASJJKrJPs18g\nIp4DXJSZp0bEM4HvAavqP748M78ZEWcBy4CdwPLMvCEiHg18FVgMbAXOzMzNza5XklRNUwMkIs4H\n3gJsq286FrgkM/953D4HA+cCxwDzgJURcQtwNvDrzPxYRJwOXAi8t5n1SpKqa3YP5G7gtcDV9cfH\nAkdExGuo9ULeBzwbWJmZu4CtEbEaOBo4Efhk/Xk3UgsQSVKHaOoYSGZeC+wat+lnwPmZeTKwBvgo\ncCAwOG6fbcACoG/c9qH6fpKkDtH0MZAJrsvM3aFwHXAZ8CP+PBz6gC3Uxj36xm17sOqL9Pf3Tb3T\nHGFbjGlnW2zZMr9tr70nvi/G2BblWh0gN0fEezLz58ALgDuA24HlEbEfcABwJHAncBvwcuDn9a8r\nqr7Ipk1Dja67K/X399kWde1ui4GBbVPv1EK+L2ra/b7oJCVB2uoAORv4dEQ8DPwBWJaZ2yLiMmAl\n0ANckJkPR8TlwJcjYgXwEPCmFtcqSZpE0wMkM9cDJ9S//wW1wfGJ+1wFXDVh2w7gDc2uT5JUxomE\nkqQiBogkqYgBIkkqYoBIkooYIJKkIgaIJKmIASJJKtLqiYSS2mh0ZIS1a9fOaGb8kiWH0dvb28Cq\n1K0MEGkO2TG0iY9c8QDzFiwuev72wY1cev5pLF16eIMrUzcyQKQ5Zt6CxcxfeEi7y9As4BiIJKmI\nPRBpCsPDw6xbt2ZGx9iwYX2DqpE6hwEiTWHdujWcd/H1xeMGAJvvvYuDDj2qgVVJ7WeASBXMdNxg\n++D9DaxG6gyOgUiSihggkqQiBogkqYgBIkkqYoBIkooYIJKkIgaIJKmIASJJKmKASJKKGCCSpCIG\niCSpiAEiSSpigEiSihggkqQilZZzj4jvA18ErsvMnc0tSZLUDar2QC4CXgqsjojPRsRxTaxJktQF\nKvVAMvPHwI8j4gDgr4BvR8RW4Erg8sx8qIk1SpI6UOUxkIg4BfgM8HHgJuA84PHA9U2pTJLU0aqO\ngawH1lAbB3lPZu6ob/8hcHvTqpMkdayqPZDnA6dn5lcAIuIpAJk5nJnHNKs4SVLnqhogr6B22gpg\nMfDdiFjWnJIkSd2gaoAsA54HkJnrgWOBc5tVlCSp81UNkH2B8VdaPQyMNr4cSVK3qDSIDlwH3BoR\n36g/fh1efSVJc1qlHkhmfhC4DAjgMOCyzPz7ZhYmSeps01kL6y7gG9R6IwMRcVJzSpIkdYOq80A+\nC7wKuGfc5lFql/dKkuagqmMgLwZi9wRCSZKqnsJaA/Q0sxBJUnep2gMZAH4XEbcBf9y9MTPf0ZSq\nJEkdr2qA3MTYTHRJkiov5/7liFgCPBW4GXhSZq5tZmGSpM5WaQwkIk4HvgtcCiwCfhoRZzSzMElS\nZ6s6iP5B4ARgKDM3An8BfKhpVUmSOl7VMZDhzByKCAAy876IGKnyxIh4DnBRZp4aEUuBLwEjwJ2Z\neU59n7OoLdi4E1iemTdExKOBr1Jb/XcrcGZmbq7+q0mSmqlqD+S3EfEeYN+IeGZEXAH8cqonRcT5\nwOeB/eubPgVckJknA4+KiFdHxMHUVvY9ntp91z8REfsCZwO/zsyTgKuBC6fzi0mSmqtqgJwDHALs\nAL5ArUfw7grPuxt47bjHx2bmivr3NwIvAp4NrMzMXZm5FVgNHA2cyNiVXzcCL6xYqySpBapehfV/\n1MY8pjXukZnXRsSTx20aPxlxCDgQ6AMGx23fBiyYsH33vpKkDlF1LawRHnn/j/sy89Bpvt74cZM+\n4EFqvZkDJ2zfUt/eN2HfSvr7+6beaY6wLcaUtsWWLfMbXEl3W7Ro/qx6X82m36XVqvZA/nSqqz4+\n8RpqYxbT9b8RcVJm/hh4GXArcDuwPCL2Aw4AjgTuBG4DXg78vP51xZ4P+UibNg0VlDb79Pf32RZ1\nM2mLgYFtDa6muw0MbJs17yv/RsaUBOl0lnMHIDN3ZuY3KVuJ92+Bj0XET6jd5fBbmXk/tXuNrAR+\nQG2Q/WHgcuBpEbECeCfwDwWvJ0lqkqqnsN467mEPtRnpD1d5bv0e6ifUv18NnLKHfa4CrpqwbQfw\nhiqvIUlqvarzQE4d9/0o8ABweuPLkSR1i6pjIG9vdiGSpO5S9RTWWh55FRbUTmeNZuZhDa1KUkca\nHRlhw4b1Mz7OkiWH0dvb24CK1E5VT2F9DXiI2qzyncCbgeOADzepLkkdaMfQJi655gHmLbiv+Bjb\nBzdy6fmnsXTp4Q2sTO1QNUBekpnPGvf40oi4oz5ALmkOmbdgMfMXHtLuMtQBql7G2xMRf1pKJCJe\nSW2inyRpjqraA1kGfCUiHk9tLOT3wJlNq0qS1PGqXoV1B/DUiHgc8MfMdGquJM1xVe9I+OSI+C/g\np8D8iLi1fotbSdIcVXUM5HPAxdRWyr0f+DrwlWYVJUnqfFUD5HGZeQtAZo5m5udxeXVJmtOqBsiO\niDiU+mTCiDiR2rwQSdIcVfUqrPcB3wOWRsQvgUXA65tWlSSp41UNkIOpzTw/AugFfl9fcl2SNEdV\nDZB/zMwbgN82sxhJUveoGiD3RMQXgJ8BO3ZvzEyvxJKkOWrSQfSI2L3gzWZqK+8+l9q9QU5lDzeG\nkiTNHVP1QL4LHJOZb4+ID2TmJa0oSpLU+aYKkJ5x378ZMEDUdYaHh1m1ahUDA2Ur8DTi/hfSbDRV\ngIy/iVTPXveSOti6dWs47+LrmbdgcdHzN997FwcdelSDq5K6X9VBdNjzHQmlrjCTe1hsH7y/wdVI\ns8NUAfLUiFhT//6Qcd97K1tJmuOmCpAjWlKFJKnrTBog3rJWkrQ3VRdTlCTpzxggkqQiBogkqYgB\nIkkqYoBIkooYIJKkIgaIJKmIASJJKmKASJKKGCCSpCIGiCSpiAEiSSpigEiSihggkqQiBogkqYgB\nIkkqYoBIkooYIJKkIgaIJKmIASJJKmKASJKKGCCSpCIGiCSpiAEiSSqyTzteNCLuAAbrD9cCHwe+\nBIwAd2bmOfX9zgKWATuB5Zl5Q+urlSTtScsDJCL2B8jM54/b9h3ggsxcERGXR8Srgf8BzgWOAeYB\nKyPilszc2eqaJUmP1I4eyNHAYyLiZqAX+DBwTGauqP/8RuDF1HojKzNzF7A1IlYDzwDuaEPNkqQJ\n2jEGsh24ODNfApwN/DvQM+7nQ8CBQB9jp7kAtgELWlWkJGly7eiBrALuBsjM1RGxmdppqt36gAeB\nrdSCZOL2KfX39zWm0lnAtoAtW+a3uwRNsGjR/I55b3ZKHd2oHQHyDuDpwDkR8URqIXFLRJycmT8C\nXgbcCtwOLI+I/YADgCOBO6u8wKZNQ00pvNv09/fZFsDAwLZ2l6AJBga2dcR707+RMSVB2o4AuQr4\nYkSsoDbO8TZgM3BlROwL3AV8KzNHI+IyYCW1U1wXZObDbahXkrQHLQ+Q+lVUZ+zhR6fsYd+rqAWO\nJKnDtGUeiKS5a3RkhA0b1s/4OEuWHEZvb28DKlIpA0RSS+0Y2sQl1zzAvAX3FR9j++BGLj3/NJYu\nPbyBlWm6DBBJLTdvwWLmLzyk3WVohlwLS5JUxACRJBUxQCRJRQwQSVIRA0SSVMQAkSQVMUAkSUUM\nEElSEQNEklTEmejqaMPDw6xbt2ZGx2jEukuSHskAUUdbt24N5118PfMWLC4+xuZ77+KgQ49qYFWS\nwABRF5jpuknbB+9vYDWSdnMMRJJUxACRJBUxQCRJRRwDkdR1GnVXw0WLjm5ANXOXASKp6zTqroZX\nf2I+Cxc+oYGVzS0GiKSu5F0N288xEElSEQNEklTEAJEkFTFAJElFDBBJUhEDRJJUxACRJBUxQCRJ\nRQwQSVIRA0SSVMQAkSQVcS0sNY33M5dmNwNETeP9zKXZzQBRU3k/c2n2MkAkzUmjIyOsXbuWgYFt\nxcdYsuQwent7G1hVdzFAJM1JO4Y28ZErHig+xbp9cCOXnn8aS5ce3uDKuocBImnO8qZUM2OAaI+8\ngkrSVAwQ7ZFXUEmaigGivfIKKkmTcSa6JKmIASJJKjLrTmG95b2XsHPncPHzD3/iY/ibZW9tYEWS\nNDvNugB5sPcpMIN5PdsfurdxxUiatUZHRhpypeFMJyM24opJgP7+Y6b9nFkXIJLUCjuGNnHJNQ8w\nb8F9xcdoxGTERlwxuX1wIz/7tgEiSS0z0ysVG9GL2bBhfdsmRHZ0gERED/CvwNHAH4F3ZubM+2pz\nwPDwMKtWrSpe58dJgFLzNaIX0875Vh0dIMBrgP0z84SIeA7wqfo2TWGm3VonAUqt0c3zrTo9QE4E\nbgLIzJ9FxLPaXM+UGjGgNTw8DPTQ21t+lfVMu7VOApQ0lU4PkAOBwXGPd0XEozJzpBkvNjoyzJbN\n93PPPauLj7Fhw3ouueZXM14C5IC+g1xGRFJH6/QA2Qr0jXs8ZXj0DP6W4V1l+bJt8AE2bHuYOz/5\nn0XPBxi8fw2PfcIRxc9vpO2DG4ufu2NoAOiZ0evPlmN0Qg2dcoxOqKFTjtEJNTTqGKWfFZ0eID8B\nXgl8KyKeC/xmqidcf+UFM2tJSVIlnR4g1wIvioif1B+/vZ3FSJLG9IyOjra7BklSF3IxRUlSEQNE\nklTEAJEkFTFAJElFOv0qrD2aao2siHgVcCGwE/hiZl7ZlkJboEJbvBE4j1pb/CYz392WQlug6tpp\nEfE5YHNmXtDiElumwvviOOCS+sM/AGdk5sMtL7QFKrTFm4H3A7uofV78W1sKbaH60lAXZeapE7ZP\n67OzW3sgf1ojC/gQtTWyAIiIfeqPXwicAiyLiP52FNkik7XFo4GPASdn5vOAx0bEK9tTZkvstS12\ni4h3AU9rdWFtMFVbXAG8LTNPorZc0JNbXF8rTdUWFwPPp7Z00gciYkGL62upiDgf+Dyw/4Tt0/7s\n7NYA+bM1soDxa2QdBazOzK2ZuRNYCZzU+hJbZrK2eAg4ITMfqj/eh9p/YLPVZG1BRBwPHAd8rvWl\ntdxe2yIijgA2A++PiB8CizKzfP2ezjfp+wL4FbAQOKD+eLbPbbgbeO0etk/7s7NbA2SPa2Tt5WdD\nwGz+j2KvbZGZo5m5CSAizgUek5k/aEONrbLXtoiIxwMfBd7DTNd96A6T/Y08DjgeuIzaf5svjIhT\nWlteS03WFgC/Be6gttLF9zJzayuLa7XMvJba6bqJpv3Z2a0BMtkaWVupNcRufcCDrSqsDSZdLywi\neiLiYuAFwOtaXVyLTdYWrwcOAr4P/B3wpoh4a4vra6XJ2mIzcHdmrsrMXdT+O+/4la5nYK9tERFP\nB15B7RTeEuDgiPjLllfYGab92dmtAfIT4OUAe1gj6y7gKRHx2IjYj1oX7KetL7FlJmsLqJ3r3j8z\nXzPuVNZstde2yMxPZ+Zxmfl84CLga5n5lfaU2RKTvS/WAPMj4rD64+dR+y98tpqsLQaB7cBDmTkK\nbKR2OmsumNgTn/ZnZ1cuZTLuqopn1De9HTiW2imaKyPiFdROV/QAV83mqyomawtq3fLbgRX1n40C\nl2bmd1pdZytM9b4Yt9+ZQMyRq7D29jdyCvDJ+s9uy8z3tb7K1qjQFu8C3kFtzPAe4Kx6z2zWiogn\nA1+v36zvjRR+dnZlgEiS2q9bT2FJktrMAJEkFTFAJElFDBBJUhEDRJJUxACRJBUxQCRJRQwQSVKR\n/wc0l0LLkV2wzgAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x111f9cf10>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "yelp.subjectivity.plot(kind=\"hist\", bins=20)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 87,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.text.Text at 0x1177f0ed0>"
      ]
     },
     "execution_count": 87,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAERCAYAAABsNEDqAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAIABJREFUeJzsnXt4VOW1/z8TcuUSMgQCiUAAle210lqqtrYKPU0FUaSe\nFvVUrW1pvZz+qraNBSpoVQKp9nhOq6gUb7QFPa0IKkGOLeKl2lI1FAQ2kkBEJyFAMiTkOiH798fa\nb/ZlJskkEBLg/T4PD5k9e7/vu/fMrPWu9V2XgGVZaGhoaGhoxIuE3l6AhoaGhsbxBa04NDQ0NDS6\nBK04NDQ0NDS6BK04NDQ0NDS6BK04NDQ0NDS6BK04NDQ0NDS6hMTemtgwjAuAhaZpTvIdvxb4MRAB\nNpumeWtvrE9DQ0NDIzZ6xeIwDONnwBIgxXc8FfglcIlpml8GMgzDmNYLS9TQ0NDQaAe95araCcyI\ncbwJ+KJpmk3260Sg8ZitSkNDQ0OjU/SK4jBNcyXQEuO4ZZrmPgDDMH4EDDBN87VjvT4NDQ0NjfbR\naxxHezAMIwAUAqcD3+jl5WhoaGho+NDbiiMQ49gTQINpmlfFO4hlWVYgEGsoDQ0NDY0O0C3B2duK\nw4K2SKoBwHvATcCbhmGst9//b9M0V3U0SCAQYN++2p5ea69h2LBB+v6OY5zI93ci3xucHPfXHfSa\n4jBNswz4ov33ctdbva3MNDQ0NDQ6gE4A1NDQ0NDoErTi0NDQ0NDoErTi0NDQ0NDoErTi0NDQ0NDo\nErTi0NDQ0NDoErTi0NDQ0NDoErTi0NDQ0NDoErTi0NDQ0NDoErTi0NDQ0NDoErTi0NDQ0NDoErTi\n0NDQ0NDoErTi0NDQ0NDoErTi0NDQ0NDoErTi0NDQ0NDoErTi0NDQ0NDoErTi0NDQ0NDoErTi0NDQ\n0NDoErTi0NDQ0NDoErTi0NDQ0NDoErTi0NDQ0NDoErTi0NDQ0NDoErTi0NDQ0NDoErTi0NDQ0NDo\nErTi0NDQ0NDoEnpNcRiGcYFhGOtjHL/CMIx/GIbxtmEY3++NtWloaGhotI9eURyGYfwMWAKk+I4n\nAr8G/g24FPiBYRjDjvkCNTQ0NDTaRWIvzbsTmAEs8x0/E/jINM0aAMMw3gK+Avz52C6vb2DVqnXM\nmrURGA/s4KmnJnL55Xk9Nl9VVZi77lpPWVk6ubkHKSycTDCY0aXrsrPLgSTKy4d2OkZJSRlXX72a\ncHgkGRl7WLlyOmPH5h6zdfc03ntvMzNmFNHUNJaUlFJWr57KhAnn9shcx/oZ+D+7p56axKOP7qSs\nLJ0RI/YRCLRQXp5Nbu5BrrtuBDfe+Fbcz0GNXV09kmBQvheDBw/u0v35n8fs2edTUPC+53rLot0x\nJ0yYSij0JeS3ZzJq1Hree+8vR/kpHr/oFcVhmuZKwzBiSYh04KDrdS0w+Nisqu9BlEYBEAAsbrpp\nNpWVPac47rprPatWXQ8EKC62gGUsWTKji9f9Ebg2rjGuvno1odBsIEB9vcWMGQUUF//omK27pzFj\nRhGNjb8EAjQ2Wlx55Tw+/rhnFMexfgb+z2769Hlt9woWsBy4iuJii5demktr6wPE+xzcYzc0yPdi\n4sSRXbo///PYuLGgbUx1PdDumKI0nN/enj2zj+h5nWjoLYujPdQgykNhEBCO58Jhwwb1yIJ6F+OR\nLy72/+N79D5DoaBnvlAoGNd83usGxT1GODzSc244PLJb99fddfc0mprG4l5XU9PYHlvXsX4G/s/O\nf6/yPZC/W1tPizq3o7XF+l509f785/vHlPeJOuaMeWx/e8cbeltxBHyvtwGnGYaRAdQjbqpfxTPQ\nvn21R3lpfQE7kN2b2sXt6NH7zMmp8syXk1Md13ze62riHiMjYw/19c65GRmfdOv+urvunkZKSimN\njc66UlJ29di6jvUz8H92/nsVZwGARULCR7S2xv8cYn0vcnKsLt2f/3lkZHziGTMnp9p+v70xTY7l\nb6+30F1l2NuKwwIwDONaYIBpmr8zDONOYB3yif3ONM3y3lxgb+KppyZy002zcXMcPYnZs89n48YC\n27f8CXPmXBnXdYWFk4FlNsdRAyy1OY4aCgsntXvdypXTmTGjwPaTf8LKlfHN19H8nc15LLF69VSu\nvHKe7dvfxerVU3psrmP9DPyf3dNPT+WRR9R3YD8Qobz8RXJza7j++kv59rfjfw5qbPU9XLnySjIy\nBnfp/vzPY86cK1mwINb1scccNWq97Z6S396oUX8F5nb/gZ1gCFiW1dtrOBqwTsTdgMKwYYOOyW5n\n1qyVbT5fsJg+/dhwBcfq/noLJ/L9ncj3BifF/fm9PnGhty0OjT6EsrJ03H5dea2h0TniiWLqC5Fu\n0Hej8I4naMWh0Ybc3IN2dIlYHLm5Nb29JI3jBPFEMfWFSDfou1F4xxO04jhBcDR2UUfDTx4rBr87\nuRnHCj25+1Rjh0JBcnKqupQXc/vtr/LuuwnAfi66aCAPP3xFl/IWjsZ9dDSm/95KShJxW6vl5SPw\nW6+dfTeO1T1oy/rIoRXHCYKjsYsKBjOOeOcVKwa/O7kZxwo9uft0jy1xIPHnxaxd+72264qKlpOc\nvL5LeQtH4z46GtN/bzk5BbijkCxrr+d1bm5Np9+NY3UPubmWtqyPEFpxnCDoK7uo6mpvvLy87rvo\nyefW3bH918EgysoO98hcCl3dmfvfy8wcz8SJy1i3DhoakoDrgBWkpUXIy4PCwklMmLCejr4bPfFZ\nxBrzuefOpy9G4R1P0NVxTxDk5h7Ejm6mN3dRweAezzqCwU96ZR3xoiefW3fH9l8HtZ1ee6T3oXbm\nxcVXsWrVDeTnr+9wTP9748bVs2TJDPLyLOAaIBe4hrw8WLJkBsFgRqffjZ74LGKNqSzrdeu+2rY2\nja5BWxwnCPpKLkOsGPy+jJ58bmps4QGq4x67sHAyzc1LeeedBOAAF100kMLCaR1e090cHIWu7szb\nu7eOnudTT01i+vR5bTWrnn56att7VVVhmpsjZGQ8hfA6gzq953jQV34XJxp0HsdxgJMgllzf3xHi\nSHNwZs16gVWrbujy9V25t47W2Fs5RJ3hJPhu6jwODY2TFUfKDxyLnbl3jQfZsGEveXl/ITf3IKWl\nA5D6pkXAQNas+ZjJk1dz6qktMaO52qty66/M21Fkls7n6D604tDQOAFwpDk4RyOirjN417iGcPin\nFBdLtFNOzgKgGVVZORKZxpYtK9iy5Qbc0VW33/5qW8RZcbFFc/NSkpOTfdFrTmXejiKzdD5H96EV\nh4bGCYB4LIbOdthdycHpSo5KSUkZM2a8SGVlDjAbGIoUwXYspAMHhpCYWEtLizuarAl4mQ0bKli/\n/m/ceONbNDbmAncDZwAJvPpqLSkpmZ6x3JV5N2xoobo6HHNtHVlp3c3BOVmgFYeGRpzoy64NL1UZ\nm7fsbIfdlRycruSoXH31aioq5rrOXQgMxJ3n0dSUAbR6jkEycAXh8DSuvdbp6SHvrQCuobX1Hhoa\nhvuucyrzhsOp5OfHzoHpyErrbg7OyQKtODQ86CvCsa+sw42+7NqIZ22d8SBdycHpCqfiHxdOAaYC\nTwIV9muASWRkPMioUWPZvt0kErmt7Rp/Tw9RPAEk7PdLwCICgSyGD/+EQ4daOHToJeAQMJWystdj\nrqsjK62v5EX1VWjFoeFBXxGOfWUdbvRlYRLP2mLtsN0K2rK8/V86ysHpCqcSDO6hocFtEXwKZABp\nwBzX8eVccslwliz5OrNm1bFqlWr+Kf0wvFbFIfv/A8AfgZ9jWQEqKiSL/dChaZ2urSNeR9dt6xha\ncWh40B3h2BPWQV8U0t0VJgcOhJk1a3Vc/a7jqQ8V67141hZrh52f73bJTCAhYS4pKeMJBkMd5uCo\nvBHVc3zOnOlR56j1pqcP48CBu4lExpGQsIuMjCays/9EWVkjNTXOZ5yYWMOtt36GWbNWUlKSSE5O\nAZmZ4xk3rp5//Wsou3YtAkYAO4Eg8BCigAYQK4u9pKQfVVVllJaO58Ybf08otI9NmwCygBKeeupC\nLr88divmW245nVdfdXJObrttaszzTlZoxaHhQXeEY09YB31xx9fdkNVbby2KqhxbX59EOPxT/M8s\n3vpQ/vfiWVusHbZXQY+htfVC8vLCLFnScX2xgoL3PT3H589fSkqKt4y6nyeQ3Iz8tjEmTPgfamqc\nz7ilJZ2bbnq9bVywmDhR7nHWrBfYtUvlmTh97R3OxBlnz55djBuXxahRDWzZMptQKMDmzX8EBnuu\nu+mm2VRWxlYc3/3u655+8d/5TgHFxT3TL/54hFYcfRi9EdlRWDiZpqaldmXWAzQ3D2g3KkWhJ6yD\nvpjx292Q1V27lD8eIEAodI79OvqZdaU+1Lp1krinLBgH8SX1VlWFqazcilQeOgRMAeooK0vv1Ir0\nr+W11w7T0uIotTVrFnD4MEhuRkbUvQAMGTKGUOghwEDxEdXVq+1xw0BR2z3OmSNZ7CUl/dm+3R99\nZQCLgBygknB4FqtWDbaz0NV5g1znq//Htz0H/70ebzXXjjW04ujD6I3IjmAwg5SUZMJhmbeoyCI5\nueN5e8I6OBZ5BccKY8fWsnGj2z9fh7/ftXpmHT1L/3sNDUmsWnUNGzcWcN55WRQVOfkN8XxXZFPi\n7O5F+P6Q3NyXOrUi/WtpaanFLWgjkbOABuBFJDpqStT34tRTW9iyJRtw+Ihg8BObDykCrqGhIcCq\nVc78s2atZMsWb0SW8B8/tK85BVFUAPtd56m5/dfFtuT8vExfr7l2rKEVRx9Gb/n5uzrvsbQOeiva\n6kjmXbx4Kk1N8nwqK7cSCt2CCLKFJCQMZ8SICv7jP85nwoTfUF2dTWrq3Ywdezrjxwei6kNJg6SR\nQBLwRWAFodAp7Nu3B9ndg3un3hGf4v+c09JyyMt7icLCScyc+R7+74D7GWRm1gP5wLkI2Z2MuJAG\n2fe2A7gHJXhTU+dRWnpe25qCwQy7JtfLvPPOg7S2ZtK//6ekpw+ntfV+KiuzaW11W1etzJr1gt33\n43IkHDeVxMQP6N8/C/gtTU0NNDV9DigAcklNbWTKlMcoL88mO7uGiopKPvjgPmA4wnFcAMT+vvt7\nqvf1mmvHGlpx9GH0lp+/q/MeS+ugt6KtjmTeIUOc51NdfT75+S+xYcNewuGf09oaIBSyuOGGeW0+\ndbA4eLAgimcIBjPIyjqLUKgeqUC7wv4/QCSifP3DgO+17dT9mdVq7YsWTYpyU6lKthD7OxBtAS9H\nyqdbwAN4eYdFuIVxY+OZbN4cYfPmIBs3Psv69TcQDGbwzDPfBlStql9QUaE4jDLc1kFDQwqrVl3D\n8OHz7WOioL72tWyeeebbUbWuYAUVFXeTnLyMdeu+2uHnE+tex47Npbj4Ryd8raruQiuOPozuVlc9\nWvP2pAXh3r2OH1/Pffd9Oa4d/PFihfnhtVgsRo7MIRx2xmtqGusZvz2fugi5KxGlEfFcA2cDmzzH\n3nkngTFjotfud1Olps5jzpyZbfPE+g74rRAvb5Djey8d+BfwASLIdwL3IRzPFVxwwYNccklWW2/y\ndetAFNFUe9wxrntMQviXAA0NA+2/1wKD+Otft7FrV1mMHibCK5WU9GfWrJUdWop9kU/r69CKow9D\n7eSPZNfTHRfLsbAg/Dv4pqb4dvDHixUG3uCGTz/d7OnBnZp6N+4ddUpKKY2NnfvUxV21mFDoDGAr\nXp99HTDSc6y1dT+VleXAdM/aS0r647UIzmP69BcZMeIccnMPMnv2+a5ZrZjPwJ2hnZrqXb/wDCsQ\nSyQAKNJb5guHh7NqVT/effc59u79peu6FUgGeT0SZttq/5N56uvTEaUhllZT0xVMn/4AF1wwyrc2\nyfMwzX+xZcu9bc89lqV4IvFpxwpacZzg6IuJdND9HXxv7Q67M2+0a8ctqD+HCMkB5ORs4emnp/Kd\n73TexyQYzGD9+uuZNGkZodCdiHvqbERpXIYI6BXIjtskLa3WPk/mGj78nzQ3j2b79gbgDwhfMBj4\nFxUV91JRId+TV19VrrODFBev4ZVX1nDJJY189auP8957yRw+vI+mpv0kJBxkyJAqnn56KpddVkBr\n67moCCkR/uqeVcKeehZJwDXs3TvPXhvAbmAYSUkmkUgLMAkn2msJMJKWllT8eRsVFZns2JFKTs4C\nBgzIZs+eXTQ2DgGWE4mcSVe/Z921hk8maMVxgqMvJtJB9y2HeHeHR5tE786u1Pvs/YKzCeEEICur\nlQkTzo3KE2jvHhyuIwjcAhSRlhYhGFxsE+9BpA/4ZoYMOYO9e4NtczU1mW3RV7KOh5BIpGzXWg/S\n2JgGvAxsAW6mpSXIX/4iWdnhsDsSawWh0M088sgyRoywCIWmud7b57rnKQjvkYVkjE+xjyfj8DU/\nx+FrlgNX4lghZwBXIGG6i+y/1TyD2bZtpn3PBbbCe9k+54+e596dvKR4reGTCcdccRiGEQAeBc4D\nGoHvm6ZZ6nr/P4A7gRbgKdM0HzvWazyR0BcT6cC7gx8/voH77ju6lkMsS2vRokk9EpHVnoD3Pvsp\n5OQUkJV1liuyCsCisnIreXlErcl/Dxs3yvUjRuyjokK5nzKQFq3LKCy8nvz8l1xW0fXk5/+VLVvc\nCisTd98LOXYZokCqETdQE1ID6stIqOwKFPHtzW84iERUSQXb55+fRF7e/cBpgAn0B/7LHncskvX9\nPvDfrvWkEZuvcfMnjYjilecVCESwrCeQ6K1G4Edt5zrrq7XHnwosJyOjkUsuSYzLUuyrm62+hN6w\nOK4CUkzT/KJhGBcAv7aPKfwKOBOxc7cahrHcNM2DMcbRiAN9lfhz7+CPduRKVVWYDRtaiEUI94Tb\nrr1xo4MbricYzKC09DS+8Y3FVFePxLI+IhS6k1AoGLUmvwALhbIJhS5FXEu/w+3qUmMvWTKjTZHN\nnPke2dl1TJmylD170qiqKqO2NgFYgxMBNQ0pUz4MEfJnIMqlBvgT8H3ENQRQTUODm1dZA/wE4Sym\n8cgjy0hJaaKpqQ5Iscf6EHBzGNsRa2IQItz322Msx2uROfyJjPUR8DQQwrJ+5Tpvof085FwnD8Sv\nML4W9yahr262+hJ6Q3FcjGxrME3z74ZhfN73/iaUrS2ILw1WIyaOhPjrixVq24N7rZWVHxIOn4Lf\nRdHVnaQas6SkP1VVJkOGjInqSAduAe/Ndi4snBwzuMFdrkM4iWDMNXkFWDVSSfaPyE4/G7EM6qiu\nHsntt78MJFFePpTKyg8JhW4FghQXV5OTs9gucTLbvvZFvFZHij3jQLwhtQvs9b1DQkIJlrUXy7oL\nUVgp9v06z7OkpD9JSVk0NYUQZeS2INT/45EQXgWVODjVHrfZPv4x4FS4TUtroaHhBuA5z3jJyZlk\nZNxPVdVQWluriURq+OpXf8OBA6PJzW3pksJQ6Glr+ERAbyiOdJxMJYAWwzASTNNUoRMfAu8h35gX\nTNPU6r6X0FeJ9VjwEtHTgaUokjgjYzuFhdeQn//XLu0k/eR2KBTdkQ7cAj52trMfHXEf7jUVFk5m\nw4YHCYfVzv3n9j3dgiPcl9PQcB1FRe76TdNx3EtrbSX1sv1eBsIr+K2OBYgCcwv5UxCe4QpaWxci\nLiiHLxHngLP2qqodHDqUiygV9/0pF9gAoNR+rfaGpThRWIrruBZRkLEyyjcB32o73tKym8TEgbS0\n3AwE2LfPYtu2AoqLO87d6Ag9aQ2fKOgNxVGD48AEaFMahmGci4R55CJhIn8wDONq0zT/3Nmgw4YN\n6uyU4xq9cX9lZUk4ETq1lJUl9dg6jnRcIYrdQi8LRa7m5R1i/PhRPPnkdG65ZQW7dg1k7NhDLF58\nJUOGtD9v9JiSGxAKBT3rVeO+9FIzDQ1uy8PiP//zJRYvnsqwYc6ud/z4elvRHAQiJCU9wcCBNXzl\nK0GefPLfsazDfPe7f+bNN6upq2tGBKh3Dc5rtY5BvuMD7HV8inyGWxEeoRHJOC/ynf8ZpES5213k\nzxGpxJsdngIsIDExg6ysgxw4kIFERh12XT8FUTAqLPcKJKN8rH39LeTkFDB8+Jls3/4+DQ132uuO\nAI8B+xk4sJE33vgBc+as4Pnnc3C+k4dobc0hHPZakuHwyKP6PT3RZUt30BuK423kl/AnwzAuBDa7\n3juIcBtNpmlahmFU4tjxHeJE3hX01q6nsnIX0upTBEllZUGPrONo3F9OThVuoZeTs4WsrFZyc2u4\n775J9vj9+O1vp7Vdc/iw93vjd80NHVqHV5Aesseu9q1Xxn3rrf+JqrP0/PMSFaTmraoKU1tbT0bG\nU9TV7SISuZNIJEh1tVgo+/bVMnnyMkKhHKTFaj3wDIHATixrGg7p6+YMtiCC1tmhp6S8S1PTRmAi\n0TzDPMTN5R5nL7AHsTyG2q8V6axyRMDrynoAqKClpY5Q6EL7+VyLlPxw8xj+JkxjER7lCgCGDMll\n9OhaWlrO48CBR9m7t4XDh+e1zTNw4AOkpw/hl7+8mP/930exrNH2uFNISXmIjIyD1Nc7Lr3W1o/4\n7GdfZMSIfQQCLXbJkXKUKy8et+vJ0jq2u0qxNxTHSuBrhmG8bb++yTCMa4EBpmn+zjCMJ4C3DMNo\nAkoQRkyjF5CZOZ5QyPnBZ2aOj/vaY82PRAcBXB+XYGivDHhxscWUKUuZPn0ZpaX9OXBgB0OG5HLq\nqcs8AQbucWpqDiFCcwROFnSGXR1XcNdd633hsAsR11MGJSX9bKXhDneVsh6WVU1Kyt00NZ2FCP6x\nQCowCyGHnwWWk5hYQ79+e2hqykR4kDqcyCWxHOX8VnucEcjebA9ObalqxEpYYZ9/CLgDWIWby4HR\nSCHDn+K4nlYgtaA+tuffad+535LZa9/bAD76aBtbttznen8xbkVTWTmCWbNeoLk5gmUtcJ03ly98\noT8PPih1pVSwQWPjLykudj+/qygudlx58bhddevYjtGp4jAM4wsIof1bxFH6WeDmeNxHsWCapoX8\nUtzY4Xr/ceDx7oytcXQxblwdmzc7P/hx4+rjvvZY8yPtBQG0p8Birc9PnpeXD3XVOfp6zHm9AqYe\n745cakllZ1e2lb3YvbsCd6lxcQGJhVJVVeYquQ5eV1SQSGQUQiyHEYLbTTIPAa6gpeVZWlpGAlX2\nWlYgwv0avBbHqQiHscO+Ntc171oc15KFKKXHERdUtf33WYhC+al9vrrvAYjLy339jxHL9TQCgd1c\neGEaO3fWs2/f2UCApqZTfffstfRaW8OsWpVNenqt77zxVFcntdWVAsjL+4utNPzPz+vK6ywwQofk\ndox4LI7/Qcpg/jvyy/gc8ALQLcWhcfzgSEJ5/T+8kpJ+ndYMihddsWbaU2CxIqGCwQ/xl+bobD7v\nffp5BtmNr16dhqrTFF0csI60tAh5ecsoLR1PKORPFKy117iG1laQbO+LgXLfeeq6JCT+JAGHEznD\nt67PAt9AlMA2RNh/iPAT7yBuL8diEoUxALESfgv8wjXvCsTt9RLCe/wzxnzDUS1iLcvigw/m0dw8\nAEfZ/MF3L0HEZTYEsY6+Dwymtnau77xkqqp2ez7v9sui1Hiu9bfN9X+uI0bs85yfnb0fDQfxKI4E\n0zTfMAzjD8CfTdPcYxiGzjg/CXAkobz+H3BVVRlbtji1mo7EAumKNdPezjFWJFRDQ3Vbkp5bUcZK\nxFu//voYSX5e4SRCeShiVfgVyq8QAXk1eXkvtXW527xZFTBsQnJg9yBKYQAwExGk9+NwDGcgSXXD\nEMtAtXBdgCM4Ld+6muxz1iIK7aA9xyuIorgSx/11A4676ULEkvGT8BbCV1j2mkP23weR6K1s3Iqo\nsXEkInrUOJfb6x2LuLjGI1nnwxEFJ7Csz6NyV4Tsv5nMzNdww73ZEWEfobz8RbKza4ClNscR3TbX\n/z0KBFrw8jQRNBzEowDqDcP4CfBV4D8Nw/gxjhrX0IgJv7Uiu2knd2DDhopOOwu2h1jWzA03PGd3\nLdzPRRcN5OGHr4gh2C0yMys8fS9gHI2NaqwgWVlnsW7dV6mqCnP77a/wzju11NSkA88gtZP+Rig0\ngjPPfJS1a69w9cg4B1EISxEhMxgJe01Csqjdgrs/8F3gHlJTf826dady7rkPM358gPT039LQkEAk\nMhwRnvfg3d1fi5DNSim8hwjmH9lzF9nv9UP4ltFI/MluRFHtBk5HLIRPcPI5/C62a5EIrOVIhz3V\nttVvHbyH8B+4PhOl3NKQ5D6lQF603/8UUQr32XMk2M+pGuFcJgMzkFyQGa65diBKuBZxrQ1m166V\nZGX9E1E2JhAhI2MEF110iPnzv8K9977F7t0fsXt3JueeG6aiohzTzGXjxmcZPNhL2rvdUZs2fWzP\nKeNKr3INhXgUx38A3wNmmKZZbRhGDl7nqoZGFPzWiuymndyBcHga+fndszo6smbAoqhoOcnJ61my\nZAazZ5/Pxo2qeOAeNm06yP79i9rO9VepVe4pIbETER5AhZ8+iRLkra0WV145j48/zmf9+uu54IIV\ndr7FIYS0noEIRiU4lyPJbRXAzfZ8I2hslL8bGiz27r0HEbaKP3DzDqq8x2pkt30nDiE9F9mplwI/\nQyr6GPZ5INV9yhChvAfxOgeRKKyleHf/7nn22GsY6nr/Ypxw2u2IG8vJ3BZll40opfPs6/yKye2q\nW2jP4X7/IaRMe3/E6mlEQoGHI2VQBttrWM6hQ6MQBamunU84/DOKip5l06bVhEJjkO6AAd580yHI\nGxosqqruxp0T4s6hCYUsz7gSsKCh0KniME3zU8Mw/gqcZxjG+8ArpmnqPooaXYIks/2fpwdFdwnH\n2NaMlxAtKzsMeLO0GxosEhKUMJBzDx/OIDV1Hk1NY0lJKeW226a6SpaEkT2TEkoLPNdKDw1Rkpdc\nksWqVdMQwbsYEbzgEOHXIQL9h/ZrC9lhu9edCNzlmm8h3vIe30PcSxORHIdb7LFOQ/IrzkGIa1Wh\ndiwiZJsRy0Pt5J3aU8JbjPHN8xP77ysQIr0MqQoUQKLp73GtUaK55D732Pc3GlFmb9tjxMo9UZFZ\nwxFF5Q7eJFsJAAAgAElEQVQYGIoTARYBfuCaT639XEQBNvrGHtN2rtStcvNOXg7Ksk5h+vT2OLzx\nvnHjjyg8GRBPVNWPkW/NKcD/Ao8bhrHUNM0He3pxGicORLj2szOqo3d4fnREXPqtmRtuWOGJ/oJa\nKit3U119fpRbKyEhk9ZW59xIZD+RiCiTxkaL73yngIkTRxIOpyC5Bm7hoXb4cm1Kyq62NShlpjr7\nOYLXvbv+GBH8BxHBGcRb2nyUb77TEQE8DuElnD4UMvYixDrZgezQy1AVZh1FdzNeoTsFcVc9ilg/\n2UjMy1JEiNf41jAR2e0vt99L9b0/xF7LswjnMt++j5H2/S5EQn/dVXNrUdxSbCskGZWx7uz61Xyq\nMONO+3+/G3C3/f9+gsEmGhr6ud73clCZmdUsWfJtYsM/7o52zjs5EY+r6jvABcDfTdM8YBjGROAf\ngFYcGl1CV6K0ukKARyK1iIAZjgjPoYRCt5Cf/xK5uZbHrXXppRZvvnk3TU1fQMI+v4BbMFVUjKC0\ntD/wb8BvcAuPiy5q5IMPlHWyi9Wrp7StIRjMYNGiSVx44f/hFXRhxMW1AxGeF9trdQv4hxAlUoFX\nWG3CCWstIFqxnIUjYOfaY7yMCOapeF1MSuiuIXbpkhX2+txWjkr8S0SE+mLaj+YK2cfGIgR5GInu\nuhkJwHwI2fH3s9eT7FtbLaLM6pEoKnV8hG8+EzBJSRnEZZct4+OPN/HBB7MRi2AnKodm+PCDrFw5\nk6997QVqah6yn0UNiYl3k5Q0vsOeJwC/+c3Z/OhHatwd/OY3Z7V77smIeBTHYdM0mw3DUK8bEcem\nhkaX0JUora7E0b/33mHcGe4iBIOUlaXz3HPn09z8OO+8UwtkkpTUj5SUETQ1KaHh7dfQ2rqL7dtT\ngKsRwtldYfWbHjK/qirMrFkrMU2L3bs/orl5JK2t1XgFnXJTqV01iCXhd4PUAbciO3cLcfsk4rhw\nxiAkt7sPRZ1rjNEoX77jQlJ1oNxCVykTv/uoDhHu/RFSWinWyxBrxEIiqm5DlMvpSGvYbMSd9X37\nnBQkjyMHCRZYi9fdtxBxqW3xrS0dUUJhvJxJCG+zqu8BRWRm7mbJkhsQ1xtUV4fJz090bUp+QDCY\nwaRJuaxadUPbPJdfHh+vNnPmDGbOPLLumycy4lEcGwzDeBAYYBjGVYjD8S89uyyNkx1dK23tdykN\nbLsmGMwgObk/4bAI1aIii6Sk+ThCawoiKNWO9U4ikQDxlOR2rKIVOHka1YgL6Wyg2F7b08A+EhKq\nGTjwl9TUNCCKSQnNfyCunbUIufwT13vKhdOK1N9SIaLSYEmgdv7KwilCdv2nIgokiBP9tAZvmK6a\nZx+ifC2EZzlkz1OEKICFiKtnMGKprEFcaLuRfeQGhKu4DhEPFfaaT/F9Nmcjyk9ZXufaz2mgPfbZ\nuGtRyT1cBDyBcDn/BQxlyJBcILqX+3PPne/5rPpqW4HjHfEojp8hdQ02ITF5axBmTkOjxxDPD14J\nDeEsHK4gPX0bkyZVt13jt17S0kYTiTgx+sOHB9i793rEzaNKo13HmDEvsmRJ+1VWnXHdu/cgIvym\nIQrBcQe1ti6npuYA4o5ZggjrQchO+mPELfUyXkF7EMm3VTyGyhyvAp5H+lkMR4Suk5fiKLHHER4i\n2X5/KqJMahAFZyBOhDGuOccgOSi5SHRUHZIvosJsVbTUdMRKcWeJL0eirRbhBArEspKCiNKYhijB\nTxDF2Grfo5un2RU1x6mntgB+l2Y1GzcutvNwHF5Mlwo5+ohHcaw1TTMPXQZE4ygjViE5y6LDHaQb\n3nIf1SQl/Zq0tDH0799ISUk/8vP/SmHh5BjZxFXk5Oy1+2scZs6cmdxzz+O89tqnRCIOidtZ2XVn\nXP/u/e/Ijvs0vDWiVBSTInyrgUJEKO9HmjMlEW0JHATOR/ZuFlIz6gbXOSrx77f2+M8ijY/64URA\nTUMsiQsRJTII+E+8Ah/77yDictpun/tZhDsy7DVvQviWtThRTNj/N9jXKhfbLYjQPw9RpHe65vk7\nYrHkIMrvQ/u9++zX1Uhi42HPHBkZjRQWfg2AkpL+rvekhHwo1PfbABzviEdxpBmGMco0zT09vhqN\nkwqxCskB3cwKX0skci+RSICaGouKCqd3hhPx1EI4nEpNzQ+oqRnMxInO2MnJ7xOJ3Im/q15HUOPu\n2GGxbdtshBdoQlxND9lnuUuNLETIY2fNToivEt71iOCvR6yKaoRDCCJKQSUYuoV1I5JcF/KNt8B3\n3hic8NWtSHzLQISTGIJksx8CbrfX9lnElfc4XjJ/rr2G6Qih7VZ0IA08VbJfBKej4DeRhp8TEYV4\nOmJpuBXYL5GEQHcI7nzPHJdckti2maiqckc/DfDcr64v1XOIR3EMBXbbJc4bsD890zTH9ejKNE54\ntE+Ax/fj91oSXqGh3EdlZelt7gopgOd0KXaP7V9Lfb33p1FVFeaOO15pI9nPPTfMhx+GqKpKQwRy\nBLEyUnGEaY5vTVl4I6f8a1blLSoQ4awEp+JMkpDd/Cd4hfVeey7VqyIVsRbS8Yb77rfn3G+Pk4UQ\n5iMRknu7fe2v7Hv6GHGLDfet8zRELBQiORW/QqKZkhCR4k7me9b+LAYg0fxDkEiw5YhLze+aG4Eo\nL7ellg6sICGhgSuu8PYNHzJkDKGQOtcbPKDrS/Uc4lEcl/X4KjROSsQmwK24SXE3DyLtUt2+9ENR\n13dEuGdnl1NcvBhV0iIcnkV+/kttFskdd7xCUVG1/f4h3nxzELKTVjtxdwc+ZT2U4RXwlYhgm48I\n34/w+v+lZ4gIcrcwPQVx4wSRnfoAvHWUDiK79Ca8u3epziu8RD/7/ERk/+fuz7EciU5SSuoB3xit\nvvsIIS6kc5BSLGn2GmMpAgtv7okK9wWnJpdfudUhLi11fQFwDenpD7Jkyc1t0WxlZel2kcNb7Wez\nyx7/FOBTmpuT0egZxKM4PkbCN75qn/9XxJmqoXFEKCycTFPTUv7+90Qsax/NzQO4556v4CfF20sG\ndBOf1dXnk5/fce+Mjgn3JLzumBUei0QsDX/uxSmI0P4zfj+8ZGsPw8mvUGP9CZiAFO8L2++PR6yM\nemTnX4ZTqXYAQlT/DCcB8XeudWxFlMJn7DmjrS5Z5wyc9q3+KLRBrr/9VlITItDnIXkjFk7pjzX2\n2kOIFTTXHt+d7BfCW2l3OOJu+xi413Xe/UhIcRPR+So5wHwaGlrIzV2JZe2gsVG5AKeTmHg3gUAa\nkQioKrxgsX59QVvv9xOxCVNvIh7FUYg4I59EPpGbEEftHR1dpKHRGYLBDFJSkqmuFk6jqMgiOTma\n05g1a2WnvIc3eubrbcpm5sz3OoywKSkp4+qrV1NePgKvsBpAbm64bRwpdOh+PxMJP12D7LgtvLvy\ncmRnn4j0rFDHH0BI4TCyQ44gxROXA59HLKVriG636i4T0g/pcTbcHvsNnES8WMl5VXizzv2FClWe\ngmXfk/u9ZETpXIWIgP6Im2sxsp9Uymy+vV5VlyuMKIpbEcWjrB+VeT7B9zxV+PL3gOc8a0hKKiUh\noYWmJvU8VnuubWk5DbgRsX6c462t59plYDRJfrQRj+LIAz7r6gv+CuJM1IrjJEBPd/KLJ9FPjjmV\nddev/5Qbb/w95eXZ5OYeZPbs8ykoeL9tjbNnn883vrGaUOgWYC3FxUE2bnyWF16Y7jmvsHAyV1+9\n2q5ltRy3sFLkuFN6+xm8AvUThPDNQFxLX8bxy5uIUH2E6N29skJUVNWvEB7A3+NinOs6VXjwJYTI\nLkOirFQUUhVCVKsM7QH2OZ/BCb1VtZfCSI7HEte4yYgC24lwF6rDYDmS8KfWXo1U9XWvUykzVWlW\nJTw+Zr9+E8ed9iQicg4jIsRf+fZU4C369fuYYcMe4ODB0XaG97e49NL3Xc/D37Nkv+s9v+KM/k4d\n6+6UJyLiURyJ9r9m12udOX6SoKc7+Tm8g5Te3r27Mcq9IOc4lXVraqZRVKRaglp2WXNHSbz66goa\nG8/CvcsOha7g0kvn0dj4S8+9SCG8g4jgfAb4hClTMpk//0ry89ezbh2IUpmECMq9iCVxDuJa+gci\njNVu/2JEYAURt9MO2hdyqtOenxcYgDez2l14UHESqtTIIkQZ/RcSzTQA+CKSdBhGXGD/jhDTIMrG\n7XL7BdGtX8chwn4M3ixuf6OqAa73VO0ot2J1Z7IvtMdz55jMRcKDt9rnyhoyM3dSUTEXOEhDwxq+\n/vV/YFnuLPgppKbOIzV1LOFwKk4Oy1VI+9xaWloGIe6xaJ7sWHenPBERj+L4A/C6YRgq0PtahAnU\nOAnQky00q6rCNDdHCAaf4dChEiKROwmHg3YhxGUsWjSJu+5az44dqQQCVViWn0MQd4gIf0dJNDYq\nEtbbQKmx0VvqQyKuPqShwZsX8cEH95OXt4aamp/hCL4CRFnU4rUOtuEVxAWIe8ZC3Do/R3bwOfZ6\nm4iOqvLngWxEdvwLEW4kgFdguzmJEYhVk4oorFmIG+l0+7zBiCXyS98Y6v8LkHBb1efiIFKqJBER\n7g8gkVAZiIvKvc53kdIoAxEX03xE6SQjFpF7nqE4QZkgSmK8PUcWTuJlgH37RtnzNgLjCIc/tdcw\nm0BgPCkppaxePZXc3FHccsuLvPFGFS0tj5GYuJ9LLhnMggWTWbDgfcrKXo+ZPNred9ptiaSkbOUf\n/6jBssaTkPARzz9/KV/5yhfREMRTVn2BYRgfIDWZE4D7TdNc0+Mr0+gT6Frpj65Bel646xg5ro+S\nkv5MnrysrSR6tF8+GZUlnZ6+k4YGt2sngCSrfYg3asnrv6+s3MrTT0/i61/f5FFKFRWjiHYxnYPs\n8g/4jo/xvc5EfPBJSA5EBqI0LMQCmYRTkHGXvb6piBVRa6/R3Wvjftf63ZyEtJMVqKKGDyGuqU/x\nVsV1V5j1u3nqENJbdfCbi5PoN8hec7l9P3k4FXnrEML+MUShqMZVrfY8+33zHEC4GfexEsQ9tddz\n3LLS7HW4K+YuAgqwLKli/Mgjy1iy5Fy2b6+lpWUewnVYbNtWwNixuSxZkkt7aO877c0r2tb23Fpb\nLb71rblUVGjFoRBPWfUc4FLTNH9mGMZY4F7DMN4zTXNvzy9Po7dxpLV+4u/X7XV9VFXtsLvqqfcv\nx6ltdAiYSlraavLyllFXl8Pevf6MaxMR6vOQhLM6RCmpnAiTUOgWvvOdxQQCFpblvvZTJOx1GhL9\n80dE0BcgFoD7XH9F2132fbitEpUBvQeJHJpjH/s1ohgyEUVwhj2eXxHtwgm/rUIU4i7cEUSidM+y\nX/t7Saj+Hwft53A/wsskIMpwrevc8Yj1pchst6vpd4igl+cv456NuOOK8OaeqD4dzTiKpdw+XodE\nW/0EUZDu+l4mkii4AuFi/mjP5Y32UlaCWJvOcXndMdr7Tnu/j94aW62tp3U67smEeF1VK+y/Qwjb\ntQzZfmic4DjSWj8d+ZP9O7+cnC1kZbWSm1tDSUkuoZB7dzzY/t8J9czLoy2xDy5FkdMJCZtpbb0V\nEWxhMjKWAJmEw39DduKDUTyEdIjbjjcvIgsRXssR/7s7+3s+IuRykF24KnLYgOy2mxBSeqE9fxki\njLcigvAZRCml2P9G4BQYDCCE8bPI7l2R3LchobmKSxhvj+VXupsRZeIl+mX3f799rluh3YeQ8z+z\n31OKcAXR2enD7fWqfIsH7Pv8EBEjdb7zW+zncxVO//J7kZiaVETJKfeUu75XLfA3ont1eK1FZSUE\ng3toaHCOB4Od95hr7zvt/T5650tI2Bl1/smMeBTHENM0HwcwTbMJWGIYxi09uyyNEwUdcSRq5ye1\nqqopLLy+zRqZNesFtmy5ElUCJDV1I2PGjKKmpoDMzPGMHLmf5uZE8vL+QmXlh8AliEVhMWLEZkIh\nReoO5pJLhlNYOIn8/PVs2PCCTagKcSoWxR3IrtsiMbGUlpb/xIkOUlHo2P+PRlwsERwhnI7wChlI\nSO2X7PO34m1r6na93I2jkFb75mjE4VzcobgKf0FCgFcjCnAKDi8SsO9tBULe70PKwwdjzDPMvvZR\nxJKrs6+5hmjlk2YfV2sZiUPQz0asBvf5FqL0/oAoo5GIMh2NKKGI73zTvodkRCm7cz+qOf30wdTV\nyWc/blx9W37PWWdlUln5BK2t+xk2rJGVK2fSXbgtkbS0Vt59d7bNcezk+ecv6fa4JyLiURwNhmFM\nMU2zCMAwjK/ilLjU0OgQHXEkaucXq+eB/Ihfst0J4bb+Cgru3A6YTk5OgV0VtYY5c65kwQLHFTF7\n9ufIz19PSUl/UlO3kZ4eJCHhCVJSatm797OIUL0WCBMI/BMnL2IK0T0t9uHUXlqAs+t29wNZhBQU\nXIZXUKchAnEgsnv/GNld+0uI1PquG4DDaTTbr93Z3aofxu/t1xk4GePn4Ozs/fzGYMSSCeKUUc/F\nq3zq7Pmm2MdVBz5VziOACPs7UEoe/okoCNUw6lbXnMoFthTHskmy1/87vNFj6twgZ53VwpIl3tph\ns2at5LXXnMitCy9cxtix7XMbncFriUhVZN2PIzbiURw3A783DEP9Cj4G2uu32CkMwwggW5zzkK3V\n903TLHW9PxGnQlwF8G3TNJujBtI4LtBdjkT9iNtL5PNbMllZZ7FunVMC3U2OepWMEki3MWXKUjZt\n2kYo5IS9RiL3tp0XCNyNZQ1EBPNpiKUxGtld34EUCbyYaMJcdeLzh+Jux4nAmmaPq0prqL7dYUQx\nVeNkev8NqUirLJxnffONQsJtgzhhr3WI66wCp6THFESZTMDhKd7A2/u80n4dxFE+bvfWdkSxucN0\nc3GUL4hS2W/fj79hlHo93H4GD+EED/g7FjbY8/+I0tLX8KMnI/40OkY8UVXFwDmGYWQCEdM0jzSs\n5iogxTTNLxqGcQHCEF7lev8J4GrTNEsNw/gu8q386Ajn1Ogl9BRH0p4lE4uMjybhRXiVlw9l/fqv\nc8stv+XNN2uIRLJc5x3EspIRPuF9ZKfvKBVRPiMQjuA0xCV1HaJYBuE0K5qHCHJF1rvXMRFRDtfa\n1z5r//0bRKAqV9YViIWh3Df+7O5SvC6xh5A92TzXsXtxyqLU2mt8BSdfRPUNmYbwOMPt88barz+D\n/AwHIy6oesT1VUd0gEAlYnG5S7Wr95TVU2yP/317zLtw8jHUuamIRTKYAwd2AF/HjZ6M+NPoGO0q\nDsMwEhAb83XTNLcA/wHMMgzjfeBHR6BALsYO4zBN8++GYXzeNed4ZPt2p2EY5wAvm6aplcZxhqOZ\nmesIfelst3ZtPeec8zD79/dHdsI5BAJh3n23gV27yliw4H2Ponn11btJTh6AZFa7Bdsf2b27kR//\neC1vvbWHSGQ2sodRvEEE4SEUYe0vUT4AsQLcAvsexG1zm73etYjgTcBpA1ttz6Git+oQpRRELJSw\nfa6/Iu15iFBfgCiA+5Ed+h5EIbmryZ6CkPR+cvtmHOWk1qwirNznTkAsC7f77T773kYgToegvfaB\nSDDBXUhUWKVr/GYkCmw2kocxGKnB9YF97lbElfYZhCyXBD55LgcQhdwE3EBFxRnceOMyHn74irbv\nkqp19u67CcABmpsHUFpaxj33vMu77ybQ2rqP5OSDNDfnkJBQz0UXDfRc3xH++te/cd11G2htPU3n\nccRARxZHAfLpvmwYxpeQb87VSPjD/wDf6eacqn6EQothGAl2SZOhSJ/IW5Ft1MuGYfzTNM3XuzmX\nRi/gaGbmOrtKydloalpBZaU7I3k5lnUbe/dazJghPIdbCDY2foHGxkpEIDUigg3gLsLhAGvXqh36\nWrwVY5/AK0yHEp2k5+8dfioi9Ny1mdT5CxAXVCHenhkL8RLhc5Eop98RvVMPIErhRt+4B/Bmad+N\nUz9LHauy5/IrpByiieptiGJwn5eJw1XMsOdw3988hLe5y3UsBeFbGux7ChBdRXg+Qsx/as91HeLO\ncyvk2bS2zqGoaDnJyevbvkuq1lk47NQ627SpwJX7owISAsBtUdd3BFEawiPpPI5odKQ4piI1qloM\nw7gd+JNpmq8BrxmGse0I5qzBSX0FUEoD5Bew0zTNHQCGYaxFwlRe72zQYcMGdXbKcY3j6f5CIa/Q\nCYWCna6/vfeffHI6t9yygpdfjlBf7/aRY//vZFGHwyO5+OJ6X7e/OkRYXoHs9L9LdHTRUETIuY/V\nEO1+WYgI1MEIN7HId85WpOyHqvXkWEqSSNdAdGKhvyf3hYgSy0AU2lBkp6+iwBR3UGQ/i8P22O4x\nxiLWi1IUexFacjSimPz3lWPPNR4h+m9D3Fju88pxuvoFiHa75djX3o9Ea6UhUVdLEStFnesvWzIG\nEe5ZOG1v/Xko6vUgQiHvd8X/XQuH/dbTIM/f/uvbg+RtOOO0tp52XP0GexodKY7Dpmm22H9fimwB\nFBKOYM63EZv7T4ZhXIgEnyuUAgMNwxhnE+ZfRrZeneJEjnw43iI7cnKqcAudnJzqDtff8f3147e/\nnUZT0wt2KRJ/eQ6nsmtGxifcd9/1vPVWgZ08WIe0k1mLI9gDJCVtJBJxZ5QHEPeJu+jeIcSt04j4\n2rMR4fY24sopQtxVKqdjJ+IKGoNjSTiWkjOuX3D7+Qp3FNNixMW2FiGx/4EIf6dul/yUfkG0Mogu\nEy8Kb5BrzZ8iyucTRLj/DdnxZyAeZVVLqg7hIhYjrqUaRJH4I7SqELfXYXsNf0KUTb3rXL9C3olY\nHN9ESqMsxKn9pc5RQQa15OS0eL4r/u9aRsYe6utjRajFvr49JCR8ZPeyl2sTEnYeV7/BeNFdZRiQ\njNloGIahsnAGIexgtmmaVYZhfAZYaprmxO5M6Iqq+ox96CaksM0A0zR/ZxjGpcg3G+BvpmnGU4XX\nOhE/VIXjTXFUV4fJz1/viaTqyK88bNggTHNPh7yIGrOkpB+VlaVUVg5BhFAYOJNAYBtf/vJwqqqG\ns2/fdhoaBgFD6N+/gvr6DGpqhiC79sFMmfIYmzZVEwqNRPz1w5GEtQxESWxFhN0we47RiCBWyW/P\n2q83I0JvEMIJfAsRviC771H23z903e0LSLjqeISfSEeUlMqjuAz5+n8RRwHUIYI6zV7XKN+YjyCW\nkEpgLEYE/5Wuc16y3yvFm+HtzidRPI2BWDbN9t+19rzfQxWjlNe7Ec81iKJ7Ea8bbT5iZeXiFE3c\nhyiqMxCFcB0iCixEMfW3x/7YfkY7EIvpbIYPP8gbb/wg5vdCfdfmzPkc99zzLu+8k0Br635SUg7S\n1JRNQkKDzXFMi4vjeOONv/GtbymOQ/I4TkSOY9iwQYHOz4pGR4rjUiSOLx1YYJpmgZ34Nx/4jmma\na2Ne2DvQiuM4xrBhg7jqqmc9IbPTp3fMi1RXh5k0yV3LSvnOvdxCTo4kjR04YJKePozq6k9oaBiI\nZWVw6NAuLOuHSC7DJzh9M2KP5SS/PYm4thoQYermK37uu+4PiHB0jxFBMs5/ipDG/RHlkGOvYyDi\nClN4FMdFtBtHSbgV2T7E7bULUYRpRJPgQ+zj37XHDdvrudk115OI5eWO6lI80E9xWr66OQSV1FiA\nZK4r3I9XSS3EIe0/hyi4Efa/WkT5jsOrfJzkxwkTXvSEXB8LnAS/vW4pjnZdVaZpvm7XpupvmmbY\nPvw+8GUd6aRxtNHVmPxgMIOsrLMIhfy+c8WBCLcQCo0kFGpGiN3F7N17Bl6BOg84k+TkSpqbn0WE\n6yHfWLheWzhZ1M8QzTeouA8V5VSP7OLPx7EoHkIE6lBkx/4J4hIbiuyuh+N11exFwmVX4HU/LbCv\nG48oo2Ykgmk5YgGohLytiKJ6wn7t5K3Ic/PP9QLRkVbqPP8zUS49y16LO5vdz+cMR9xZKslvGqJM\nvokoxtUMHPhP0tOVst9hl8sHHW7bt9BhHoedeNfsev33Hl+RxgmJqqowd9zxit2CNZOLLjrMww9f\n5uu50bWYfO81yneuOBA/t7DCLoDnJ2fHArVEIoOR8NrTcXqF+/mUfyHuo2achkm/Qfp8BxFBuNS+\n3h2hdTdOiOl/IaXPRyO5Gf+w51R5FABfQYT+YSTPQpHDfqHtj7B63P57qn3/DTiZ5n+w5ylBXEL9\n7HFm4ii5fyEk+v8inI4/p2KFfb27NewB+14CePNcFtnz+wn2Ib57ONv1WdWRmDiG4uJvAlBdfQH5\n+S91u8CmRs8hnsxxDY0jhpRQT0eFjRYVWVjWUlJSkgmFggwdWseUKUspLx8aU0jEyg1xZ6VnZ9cA\nS3n77UZqagrwRvIEgAGkpb1BQ0MykkvxKWJ5yG7esgbjhG424xQpvAsRvLlIJnYr4h5yly13d8Ir\nBz7rm/sLCN9gIYplOeKWakIE8idIpnktYpUMtscrsP+pulG1OBnlKYj/f4X93lSc8uSq5MhCe92Z\niHJT652N8AplrrksRIlZOG4wd0fD79vvPYyjNPcill2DfT9u66ufvV538cg6omta1SHW0rNAMzU1\nrZ5GXspdWVXl5jJ0177ehlYcGscE4nrqh1ugvvtuQlsMvuI12vNht5cb4hYsd921nkgkGRGUrXgj\nu7Zw4EAK3v4W85HSIUWI8ByECPMIEj10AKH43H56VbLDq5QEFhJ51Ui0cFTnHsbJdVCtY911p561\n1zUGJ+fiS4hL7VScarZBhItQSmspIsCfQZRiFsI9rCfaZfRFe003I5ZBFuJ+uxPJM+mPV6G8hwQ3\nDkCsH8Wt3G0/i6uQ8F1/tNds36c4DkkFm+1aw2VIT48k4Ce0tgbaGnm5OS7dta9vIZ5+HFuQb+My\n0zQren5JGicixK2UiFegems8dcRrdMaBOIJlOSLglLtnIOnp21i//nrOOOP/8ArQ0xABrLgLRdC6\nXS7P+K4ZSXRI6d+QKKQKnIqzyu+/E/Hpg5OHocZbi4S7usdvQaKmtiFuHQtJY3K7vp7FcVEp99Ve\nvPcO+cQAACAASURBVElzc5Equh8hStCvyAba9z4cIdzvtddwOmIBzUWsmizEAvMXH7wWh8hegURx\nrfHdy+ft55mDKKZ/R6yh08jIMIFMPv/550hKyuD11wfQ0KCuPciGDXvJy/tLu2VjdF2q3kU8Fsfl\nSGrresMwSoGngFWmaUZ6dGUaJxQKCyfT3Pwy77zzIMJxtAIDKCqKj9fojANxBMtUxId/BWrH/KUv\n7bWzjEtpbHQL0JD9/wdIZNU4RHB/jFMldh9eoZuA7OwX4CTv3YVYLWORwge/tc+bgxO+GkaIYXfe\nwQCiK9b2R9xMsxGyfC6iSByhKoL9JUSBKR7Bn0g4HrE0WhAX1DbEZebPbUnBCRtWOSB341T4DRCd\nMKkUbQXiYvsEWEd0k6s6+/524FVqO9mxYy7gWIopKS00NKhijGsIh39KcbFjXeTmWrouVR9CPEUO\ny5C4vPsMw5iBlBt5zDCM3wP3maZ5oIfXqHECIBjM4JlnvEWVq6vDJCe7+3G0T352VmXXUSwZiMKY\nh+yeP6W5OZmqqjATJ6by1ltzsazTCAR2kpaWTkPDXCwrkegy5V9ABHMyTnTSRpy2rqsRq6YIeAvh\nC1Rnu2ZECQXw9vU4hFTqUa6onYjbyR39pLiTz9j/n2MfV9FKdUiE2FrEtbYV+DGiANwRTck47WAL\n7PmqEVfZr5HaUCsQRfucvT53nSl3xz2/cjMRIv1WnKCA+4D/h5dor0IqCL+NKMDTgJ0EAoqY97dr\ntcjIkI1FOOy1Lp577nyOpBMlQElJGVdfvZrq6pEEg3tYuXJ6u2XY1bnh8EgyMjo+92REu3kcCoZh\nDERszOuRbc2zyDft68B3TdP8fAeXHyvoPI7jGLHur6uFEt2JYB9++L6nPHpGxoNccslwj4DKyVE1\njRYjBPV3XaOtxiGz70bcU+kI1/B/iGIIIcLXHdqryHXVCMn9nsrtWIgkvzUgbpwzEAWwBVEaqtf4\nPMQ9Favm1Tii60T5o7h+at9LEaJsSuzrvh9jbXMRBafqSVXjralVjRDip+K0m30QUWqqsKJqBete\n03AkSksR5sXAaBITG7n88lQKCyczc+Z7FBc7xbEnTHiR3NyDrFp1A97PL+uICfEJE37jqWOVk1NA\ncfGPjvjc4xlHPY/DhV2IPXqvaZpvqIOGYSwGvtadSTU0OkNXyVB3BM748VWeHStkRvnInV7VdYj7\npz0yexwi4M9GuIxShJdIQkhw9zzuukhT8baYVY2QDCQdKgsJeb3OvubLSGjvefb8KjLLH4IbRKwT\n9zF/scUcYhdaVIEBqklTBBEBqgih27LKd73+EFFEg5HyJo8hyrTVXrc6rqyNQ0jNrk14CfNa4Fpa\nWhwCPJYLSlmXGza0EA6nEg7PYtWqwRwpId6V/uTd6WV+MiEexfE90zRXuw8YhvEN0zRfQMI6NDSO\nOo6EDL3oosM2dyL8QmsrrvayQaAay9qBWBa1iDVxD5KPsAvZOauOewmIZVGKcBhnIy4q1XfbrXCc\nulmygz8LEaJu4b0DsQ7uRdxL6vrBOLt5kOz1WLkkB+z7ctfa8te7CiHJfiEk2kr11lDchArXXYEo\ntK1IdvpQxPI5E29jJuVyA+Fg3EmIi+xnkoxwJ8piehYnAVL1EfHm0CgXlL80Oji95N3WyJES4l3p\nT96dXuYnEzrqxzET+Wb/0jAMt32YhDB3L/Tw2jROcCh3lHAcVR5XRFcTAt2urexsmDLlMd555xDh\n8E+pqQlQU2ORmHgfEOHw4WQaG50mSamp8wgETqWhoRxx5TyOcAFzcASku7SGiipqQH4KZyF1p/rZ\nxx5DBPx+4Ha8HMZoRPAnIa4p9d4/EOGr7nkKIpT7I26fHISob0FyMtR17wI/wGst/ARHgC/H4Try\nEaE/DhHiUxHl6He53YMorkEIz7MdUSjX4pQ+UVV/cxCl9P+QUGFVFLEZb+fAFfhDpLdt+yfh8Gm+\n0ujVbNq0mKyss2xlPwEJXBhAZeWHVFef73FXdcWluXLldGbMKLA5jk9YufLKmOcBPPXUJKZPn0dT\n01hSUkp5+ump7Z57MqIjiyMdCbYeBLiZqBbEKaqhcUTwE6NuV0R7ZHh7gsLv2rrssqX48xdaWkYj\n4aMve46fccb5tl/9Zpy8Cu85XjfUQJx2q19ErIpbEdJZcQ3ViAJ5FaeVqoEoD1UBdy0OR5CJWBwO\nqS+C+m84xPtoZAdfgERCbbXHeQyxErYhri7VY9y/7jMRAvxbiHJ8036tlIE6r589t4oIG4vsIf8b\nUXgWsTLzpaPhIft+knxjqpIoqu1tEk1NP2HGjMW+HiprCYVm26VkppOaOo/GRnmmodAV5Od3P79j\n7NjcuHmKRx/d2TZvY6PFI48sY8mSc+O69mRAR7WqlgBLDMP4qmmafzmGa9I4SdCRO6q9lrPtCQr/\nWJJc6HYFuXMovO4f5Vdfs+ZRIpHTYp7jdUP9HSlzXohXcObiZE9/itel8xCyF/s+TqeAH7vev9e+\n9hRkZz4UcR9dhCiH7+Elzr+NKJ5P8Ia6qnLusdZdhWTUBxH30sXIbn4wTl/ywTglRFRipBprN5KP\nsQKxrNyWRwThQmYh3MY7vnUkIlH9botNuISJE93WpZe/CQS8/I3fXdVT+R06b6RjdOSqesI0zR8A\nvzAMI8rCME1zco+uTOOER3fqU8X6QVdVhW23hrs97AFEiK1AssGTcXpaT0VcOGFSU8spLT2P/Py/\nkpbWQiSy03dOLeK22o0Y258iwt+f7DbAPkeRwX6LxUDCVhchoasv+t4fY6/RvdM/CxHQd+D0Jg/g\n5JhAdFb4YIQEz0L4lDNwSqoPRrgaC4lw+hBvGPICxOpRdapiEfP9EMWmeBW/5bHQvpdhOEmQ5YgF\nloYEBqjKvxaDB39Mc/MIMjKeAvaTmtpIRYXD3/i5Bv93pKf6jut+5h2jI1fV4/b/9xyDdWichFDu\nqHjyOBRi/aCFJ7kV5efPydnCeedlUFSkymZUk5OzmCFDcqmsvJ+qqqG0tlaTlHSIxsYH2Lw5wObN\nFiNG3EdNzWFE+J1tz/EtxFpYhCPQXye6kdF7SEjvX4htsaiKu6qm01akVId6fx8i5GPt9BV/ASKA\nSxDFk4QIevc8exFFMxjhNJJx+JZ9iItrIaIA3LkaAaQnyFZEAc9H3FzugobJODWrforwO/5ufVmI\nlfQnxC2o8BLCtXwO+DUJCWPJygpx9tlpFBU5pemnTHmM5GTHRTlnznQWLGg/f6Oz/J7uojvfzZMJ\nHbmq3rP/vBNYBqy2q+VqaBwVKHdUV/JUYgmKmTPfwx0FNGRILdDQtou96KJBPPzw9VgWTJ5cRkvL\nKcBgmprAEXoHqatTJTYOIJFTimQdg1egH0QqyN6HRCylIuGpo+2//RaLIqIthGhORDgRd7hrA04p\nE/9OX5HelYhV486veARvfaqh9jpT7etDSOmSiD3OtxGlstBeh1vp7ECsqz8jpPRkvAl9t7nWZNjr\n2eMbo9K+JtN3XH2+o5k+/fQ2N2RenlK0Mm55eXZUvbIlS9pPvGvPpXmk6M5382RCPOG4TyC/lv8y\nDONV4Pemab7eo6vS0GgHsQSF1wqpZufO7WzZ4iTEJScvIxjMYNaslZ6kLm/00BZqa90JeO4oKlV1\nVgn0ImTn7Y64mm7/X4UI5WHIDv9axN//BtL57wcIYe5WDImIMN+Bk2vh3um/iwjqZPuaFxFFE8Fb\ngHE2QoAr11ED0QmKSvmdgcS8LERcU8qaGIxYWPuQaHtV6PBt+z3s16UIt3LQHvcQwrd8D+E4vgIs\nJyOj0S4vE6G8/MUoq0C7hI5PxFNy5BXgFcMw0hD27CHDMIaapqnz7zX6BNxWSGXlVkKhicQiNktK\n+hO9k1eC9WLEO3s2sjsOI2U4TKRUxjxkFz+NaIugEdn5D7Gv7Ye4kyYipHMQsSrGIEpEubzclWQX\nIYosH1EI7nLkTQixvAJvV76nfOv4DGK1qGP+3iODEAtkt70+pViKkFIoCgaS2KdKy9fZa3gAsapC\nOAS6KqnyEqKARiPk/yFGjChlw4YbO8z27ilXk0bPIq6y6oZhnIVsY76J2KYP9+SiNDS6ArcVkpcH\noZC3tpLaxVZVmZ7jgcAgLMsJA3WK+qnIpWxEGF+OKJLHkd14A16LYC/enI/5OO4klQyXhISjpiHN\nk7wuGnExzUZcXiEkBDcbCbG9CCcM2HGtSeVbtztoq/2/Ouav4qvcZkuRXA/lsmrxnXfIfq8WyTVZ\ni3AZHyIJioMRS8c/dsi+9k5GjHiEz342k5kz3yM7uxxIsnuteHMt4nU1daXOVLy5HV0ta6PhIJ6y\n6puRb9bvgcmmaZb3+Ko0NLoJcX1ciZsonz37SmbNWkl1dQ5O1A8MGhSmpiZ2GKiU/PgGDjldj5DH\nAUSYP4C4o6oRN48/Qsr9WikiN9mtuBB1bBeiPOrt/3/qek+F2LoJ9zX2etxlPs5GsrdVfa0QjtVw\n0B733xGFoupHNSLutccQfqIRUW6LESJ9LY7r6wpEcZ6CuNzyCQTSCQSySEj4mJaWHyARX9DYmENR\n0U1AgOJi1cO9+700rr56dZubsaHBYsaM9mtHxZvboXt8dB/xWBzXmaa5ucdXoqFxFCCuD9VuNExh\n4fXk5/sTDaWG05e+9BibNhUQCp0DbMZbxuN9pE94LiJgP0WiplbghMcOQITuJ3iVgN8ScPfgUJbD\nxYhraqx9fTOSJV6F5Hu4zx9pzwtOlVkQ5eNPwgsi7qJkRPCrUGJ3rsenqE6MYjn5s+KvQRRXEtEK\n1bCfE0A6lvULLCtAa6viUHJxwqHdLjOnO+CGDRVUV4e7tLvvSu2oeHMwdK5G9xFPHsf/GIZh+d/X\neRwafRGxXB9+AZGWFiEvbxmFhdMAuP32V1i3robDh+chvEQd4sp5HHFfvYFjRQwkeheuqtiqnX82\nYmGMRBSBv/e2iZTwUGGzs/EmE97tO38bQmZ/hAjgRET4u0uWqFIjFpLlrTiRMKKI7rbvoRJvfw/F\nfai/m+w5VIc/1bbWHx1l4e8BkphYy5ln/olx4+ppbnb3WqnBzemEw9OiMsA7Q1dqR8VLuGtivvvQ\neRwaJzz8AiIvDxYtmtRWhr2y8gCHD5+HeGTd9YuyEOF7Dk7oqioW6Ba8uXh3/ssRK6IfYgFkIrxC\nIkIR1iMKSe24/bkQI/AqonMRobsYUQI7kOTAta5zMpFoq51IyZIXEYUTRKyOsxB32G32OO0pg3L7\nWgtRIKqSruoXrvqIlyKKwxmnpaWKDz+EAwf28/TTk9ryMbKza3j77RRqapx73LChxdPhrzProyt1\npuIl3DUx333E04/jN6Zp/sh37BnTNG9s75pegO7HcRyjp+/P3atDCYgf//hV1q51Es+EC4Doyq8/\nRyKGvoy4WkByGtzk96+QnX1/hCPIRhSE2z20AHFLqQ6F7nnm4s3gno+3fa1yHymX0m6Ek/CPn4Ak\n9d0Q49rH7DUORyKqTkEsjypE2XwBIb9/SOyQ5IcQ95U6vsW+n4OI0irBXVzR379i1qwXPD023GNP\nn953uYWT4LcX6PysaHTkqvodUkbz84ZhnO16KwknoLvLMAwjgBThOQ/5lX3fNM3SGOc9DhwwTXNO\nd+fSODnhrZQbO6JHynj7eYTtSB7FMEQQqoS3GuQrr3IaDuFN3rsFJ0JJhcs+7Rt/FE7vjTF4LYog\n3vDbMOImykAE+zjEnaWufwexWNQatuDwCKqGlJpXWUdVeCO/5tlrSsThOsBbILEZp1xJhut44/9v\n79zDoyrPBP6bkBADBBJiEKI04iUfWq1U2iroyqqVKl6AtdXarnXtSltsfbbttigo1pYKmtrWXiy1\nUFu1FdztSgErFkWkyloXtShe+EQuUZxwTYYEyH1m/3jPyTlnkkxmcp/k/T0PD5k5t++bM/O9571T\nWAj79/tzWX4XODbeB+F/ut+1axuRyKzmfdW3kH4kMlX9EPmG/wwxV7nfxkZEB+4oM4Bsa+1kY8w5\nSEnRGf4djDFfRewDGzpxHWWAEoyWaSui5wDB0Nx3iMX8T/2/RQIJTwA+IBS6jVjsdMS3cDPBhdT9\n21/Cw+1V7uZsHMErJBhFhIB7rVuRxXkM4lR2tY0f07KlbRliBgvh9ctoQp7BxiEFGONDa2NIRJVf\noJyKV279h4hgGkIwzPhdJBR4pzNngBjZ2e8xevTZ7N/vP5/bv711H4Tf9zRr1iGnMRNAJfv2vc3U\nqWhIbBqRqOTILmCXMeZ84EvW2geMMccjuuw/OnHN8xHjLNbal40xgdazxphJiIfyQcQjqChJ4Woa\na9dCW0lw7tPt2Wdn89xz3lP+oEFjaWz0L4RNiCDYzXHH1fPLX17EF7/4PPX1wwlmULsaQrxwGIFo\nER8SDK29AxEQ85HoqHeRRdtf+fZpxEkdjCSS57itzrkfQTQjt5fHV53t5yPCpggRMrnOvmUEBUrY\nd163UOJQvPpT7yK+mCuQPJU1zZ9VVlYmJ510hC1bvPMNG9ZEY+N8QqGTyc8vT9oHIQmbUka9oyGx\n3ZGPkahXjJJcOO4fEaMuyK8kA6lddXUHrzkceQxzaTTGZFhro8aY0YiBdwaSJaUoSeNpGv5IoGAS\nnBs5k509OO7ooAYix50AfIy9e9/iG994kfr6k5DF9C7EIb4Vr/+4v9/2jxEt5H1ESPgX/3Pw+pkv\nxytsuAwxK0XxOun9Pm5MmYgQWuac+xa8yC7XH5OPCI1CJB+jHCk2uBSvle02pDQIeELkNMTM9U3f\n9W5zxpiBZyaD2toHKS29iPr6B3n22YM0NBzP4cPFwHSmT1/NkiWJe160TNjsXEhsd+RjJOoVoyQn\nOIqttVcBWGurkDLrmztxzSq8+D+ADGtt1Pn7c8jjz1PILy7HGLPVWvtIeyctLMxtb5e0RufXPuFw\nPvJDl0igIUMauOSSekKhR/jww5GMG3eYu++ewje+8SQbNgzFvxjGYncQ9DP4W75eyYEDi5BoJ9dJ\nPoysrHIaGopoaQZyS6gvRyKR/N30anz7DaVlX273uOsQq7B/TA144bPj4q5Z5M4E0RAKCZY/H4KX\nGV8J/BRxkLsJjNMQ81z8PC519vUEWDR6gPnzXyAUyqShwe83eYRweGRK97Kk5Ggg4q2kpCbl74J3\n32Xc4XB+p79P3XHO/kQygiNmjDnTTQI0xoxHvsEdZSPy6/iTMeZcJPMKAGvtL4BfONe5ATDJCA2g\nv0c+6PySoKioAlnApKf2JZc8ypIlQcV11qwVrWglMZqaiikq2uW0LN1FOGzwLxyhUAGwB3H5jQZ2\n0NBQjTjHjyPoG9iMOLBHIwv3bN+2XzvXHop092tEtBb/gu2WUB9FMDx4NZ55zO0b4p53G2Ld3YEX\nsfUUYg5ziy66WeQjkKLXI/CirpYjAtA7Z3b2durrf0Qs9l38wQDRaD7/9V/XkZn5m7hxxygqqkzp\nXi5Y8E/U1XkhsQsWXJjyd8G77x0bQ0+dsy/SUWGYjOD4DvCMMcb1yB2L1GbuKCuAS4wxG53XNxpj\nrgOGWmuXJjhOURKSTFy+lww4DTEpGUS7+ByjRj3P2rUXU1k5kSlTfs+ePd7CMWjQBzQ25hAsHXIP\nski7FWIPIWG4n0eEwqXAswQX1wNI4p5rYvoxwV7jMSSZ7zBSe8qfzf4PREOJOMfcgWgeOxAtZCci\nbNwoLVeLqUSEyhmIUHN7h4MXdeWe/33nHNsxppE33rgQf8l6ic5qBA7R2LgvMO6srA8pLf1SwnsU\nT1eURe+OfAztx5GYdvM4AIwxg5EspAbAWmvruntgKaJ5HGlMT84vmE/wR/zRTUVFixg16nSKiw8x\nb95EFi58jbKy4Wzd+iq1td8GXkQWcrddag1S9mMashA/iPgDPovkVRyDaBO34ZUo2Y0XoQSS03Ee\n8kR/LuJgvxTxR3wBeAJZ3HORjPAZeM7xTGShdwsRZiNRVWchms4sZ1zLaJmgGF9i5E7Ez+Evwz7X\nua7fHOWGEbvmPHds1Vx22SEefjjxM2W6FRYcAL+9rs3jcDHG5CP1EE5GfBCLjTH/aa2t7MgFFaU3\n8T+djhlTBfyW8vJjneie2YTD+S0crOJQz8crMhjfLvXHiEsuAxEOP0UW1DsRk9BPkLIkrrDyaxdD\nkKZQ5xI0S00G/oJoFn7Hu+v/OIqE/J5AsATKTETonI4Ipdm0rDdVjwikzYiw+CHSmS8zbr8S53yu\nmeoV4Fvk5S0FCohErsFNiszL+5D77/98u5+/FhbsHyRjqloCrEVSS6uRMI0/IDGHipI2tHzavaL5\naVeie7wENn90j1cnaRrypO86qSHoDHf7VWQiebJPIgLndN/+lwP3IT6HPYi2sAov38IVEEcQ05K/\nWKCb0OcWETwZESDxguF0RAhdiWg+dQRNXtlI5d8QojUVIBFiH4kbw3v4zVTZ2fspKFhMQUEJBw9a\nIpEYblLklCmVSWkOWliwf5CM4Bhnrf2NMWa20zr2dmPM6909MEXpahI97bZW8M4VNCNGnEJFxXxq\na0cjz06t9a8IIb07rkKiqPxRUvOR6PUQYmZqQhbrkYgpyS1Y6PY6d81VbomT+AKJ1Yjj/QuIIHuL\noGA44hznjulMxNx0MqKluGaqzXi5vZVI4uF2xNlfhggcV6hdRn5+OeHw7U74bIzRo3/I2LET2vUB\n+AX2vn1vAVNw81biCwummylroJKM4Gg0xoxAvmkYY05F4vwUJa1I9LQ7d+5ENm1a1NwoaN686XGx\n/NeQl3cfkchcxEewHFlY3VBW/4Id33kvvmjhR5DF/01nn6FIvasipE3sOMR8lOXseztiSnoPETwh\nxI8SQoTPVYhZrAjJyfi2c15X0LxJsLaW21PDzSEByCcj4zQKC3dz8OB+JxDAy1o/5pg7OXr0uMC8\nDhwYxdix7nXaJvg5Tvf5klo6stWUlR4kIzi+BzwPfMQY82ekBsGXu3NQitIdJCqjvWjRa4FGQd/7\n3m9bqWdVgDyZ70I0gzpEWDyBLOpnINpGvNlpN1L3yn09H3n2iiJ+g+OQZL1iRBBMc84zAtEOnsTr\ngfEYQf/KFkQw5CGCJgPxbRQgJdRvAV6Im0cJImyC/paLLorx2GNzqKyMcNppzxCN+sORTyUjY09g\n/8bGajZtcsuWtL3AxwvsUaNOZ+3ai5PaV01ZfZNkeo4/bYx5BUl5HQR81Vq7t9tHpihdTGtahUv8\ngrV2bQbR6F78C+XEifW8+OKPqKvz14+6HdEw/O/NRyKSxiOlO05EzFAGCXn1qsiK9vIOoiW4ZdIX\nI76LyxFHdz1enshlyLPcOMTE9Q1EwMxz9rvLN467kFyOeoKC7C3nb4uYugYDu1i/fjhjx96DONzf\nDxxTW7uVujo3F2Q84hvx6nUlWuDbEtitmaW0R0Z6kKg67lcc38adcZsmGGNAHrVWW2vf7c4BKkpX\nEa9VLFz4KEuWFAMtF7doNAfxO9wLfJS8vK0MHjyMurpTCD69n4LnLHfDdE9AhIlbs2o5UktqOV7G\nt1ti5EPEUR7fHGou0pc8hvhEliECYDBe8p5b2gSk0m123NhOdf4+ggi4QkRLmu28fxApR7IMuI2m\nphBNTV6IbkaGdBuMRncSi32FWOxEIEZe3n0MGdJAOOyeJ/EC31aeRWtmKe2RkR4k0jhCcf/Hczzw\nV+SXoCh9nkRmEHfBWrsWamqykCf7PERLqAYKeOklkKBC/9N7OZIw11qYrhs66ybZDUXMXPElRuYi\nWon/JzcZr6bVYue9DN9sQohwWo2UMnGjrOJNZLf6Xt+D+ExcYZblvB/fmGoYkE929niMGc7mzYXO\n+GT7iSeeyuOPT2TOnNVJJcj5k/wqKrzeKLt27UGSJvOa70dXJAQq3U+i6rgPOv9/3xiTheinDcA2\na20TQGstZRWlp0k2EieRGcRdsCRB0Fv8s7L+QUPD94lE3BwM9+lfkt4yMo4ybFg5VVXLkNLmrYXO\nHkZ8IxsR38PRuP2ORyrcghvBJOakC5BFdQ/BsiX3INpQLsHS6JchAqEW0T6CZVNyckbT1PQ69fXl\niC/lX5z9dxMsmSI+mvz83RQXH8/mzW73Q+9zcz+vVBPkWhYP9JIR1SyVPiSTAHgBkrexD3nkyTXG\nXGetfcVa+5/dPUBFaY9kI3GSMYPE77N9u+HNN/05GAsC+xcUxGhoyEEWwm0En/i3IgJgGFKj6ixE\n04jv430ArwFUDDGPfQ1PgzmRoKD5KF5HPve9Y5H+aGMQ30UM0SxmNJ936tRBlJVNZvPmGXjO9+sQ\noXYvUEIo9AqDB4+loGARK1ZcRV7eCOrrn+Sll+4DCpg0KUpp6WfauhXtEq/15eXVcuKJf1azVJqR\nTFTVT4HLfUUOP4F8Qz/VnQNTlGRJNhInkRnE1Vq2bx9ERcVuRow4mU2bPuDwYX/ORgR5dtqFaA6D\nqKs7hqqq7yBP7nMIPvHPQhbnHyD+BTdM1+3jfRR5HguGuYpgyEd8Gvc6149PDhxLsC9IhXOeOuc6\n9UgsyyNkZtYQCh1k3boG6uqOONc9AjxCbm4Dubn7GD68kKqqrRQUnM1JJx0JJEf6y4j4TU3FxYd4\n6KHpznWCmt/o0fsJhRopLx8T0ALjtb4pUzJZsuTiFuft7fyN8867mm3bzkYi0Czjx7/M3/62utfG\n09dIRnDgCg3n71eMMUkdpyg9QVdE4nhay3JgbnOSm0QduYl5ryDVd9yEuZ9w+PA4RIPIwcuyjhAK\n/YJY7DHE1PQfSPkQtzeIVO+VMuaHEPNXfEJhDLEMF+PVjZpMsJbVbc643KQ9fw916QuemZlPY2Mu\n8HkaGtzmUiHgJuAezj8/j4cfvsWpGizz3rIlxuHDD/LOOxXNEWgrVkxn3LjiFtrd7NnL+eUvr4j7\nDP1jmBHQAlNxlPemr0OEhtfTfevWub02lr5IoqiqC5w/txpjfo18yxuBL+IZZBWl1+mKSBxPEhsn\n2AAAGu1JREFUa4l3FI90/rnFDd1tTwM/cHIdXL+Du/iPIBbLQYocuj0/LkeEziK8FrMfOK/d6rpV\nzjU+goTSZiHRUG4Y7k+BTzjXvhXJ6r4e+WlGkHyPakSjyQXOpbHR3zTqOrxWOCHgDMrLm+LmL9s2\nbAjR2OhFoM2cuYjNm29psd/OncMA0TY2bNjbyhhkP1cLbEvr63v5GyW0zH1RXBJpDt+Pe13q+1ud\n4kqfoSsicTytxS1k6H/6jzp/f+jbFl8fKpeiokUcPjyaqqpjkCd618zk9h138yNmI9rJI86xeYiA\neQjJMi9DIrXG+q6Rj4T++gsh1iEZ48cQrGC7zDcu93/XUV/t26+aMWOqmDVrBTt2BKPFmpr2BY7f\ns2c0lZWRFtrduHGHAdEYIhF/i1x3DJCMFtj38jdcP5E7H8068JMoqko9VcqAwdVaxMexiBEjTubQ\noe2MHFnM2LFSRXfnzgJ27bqTUOhkamvfJRbz6kPl5dWwefMtTJ26znE+g3RJno+YsfwLu9vXO35x\neg8pUBhFzFfxNajeCewfCu0iFrsbecr3C4lqZKG7w3kvRlbWG+TkvEN29iHq6saQkXEfkyYNA7Ic\nE5Gr9UiuSCwWCVwrGt3LnDnrW2h3ixdfRVNTS41h+PBazjuvjvLy5BzffS1/Y/z4lx3zlPRfHz/+\n7706nr5GMlFV62lFw7DWXtQtI1KUXiBVreXqq3/OCy/ciaQx7eTMM7OB+CdnN4HwTIIL+1hgL4MG\njaWpaS5SiDAEfBcxQ81DTFUgGsUnEd/GV51tY8nNraa6OgNZ8INaUkZGGU8/fTUPPLDatxB/ifz8\nPF8QQD6vv15GZeUYRGBMQ7Se1YiwGkwodDux2Hik/tUXKCv7R4vPaeRICceN1xguvDCTJUs+l/Tn\n2dfyN1xHeH/vx9FRknFy3+X7OwuYjngGFWVA4Y8aeuutCFKbaiSQzdatbwPBJ+ddu7YRiZxDy9pV\n1biZ2kH/A3g+lkxEY/DXqQKJ5ppNdbXfJDTN+b8aGE40+l0eeGB1qwtxfBBAMFnx8/gd8yNGnEIk\n4jm7i4s3tPnZ9DWNQelekqlVFf9tedYY8zLyKKQoaUNnS3YHo4aO4u+Wd/DgIkCenO+990JuvXU9\nu3bFnP3c8NsG5NkrmJjn9RiPIQULzycjY5TjeI/3uRwbODYzs5po9Ami0TKk9ugIYA1r10q3w/g5\nthUEkJNTRyx2J7W1pzljvZRJk/6HwYOTEwZ9TWNQupdkTFUf8b10g8wLum1EitJNpBLy6Zl0hlBR\nYRk58kTef78cr0RGsHR6RkZ+87Hedd5HIp7eQ5zdYSSnIz4BcJNz5BHgaxQVLSYabWDPnkqklMhc\nxCS2G/F9PIHkilzG5ZfnMHfu2UyZ8h51detxa1/V1FzDypUjWsyxrSCAqVMzKC39ii+XYjWlpZ9p\nNm9985t/5ZxzngEOMGnSMO6//8pmgaQ9NAYeyZiqNhD8lu9HajUrSlqRSshnfE5COOwm+LklMtyc\nDNk+cuSBVq7zv0hwovvTWYpEUh1BSoRMQExDbsc+obLyBCZPPsLevfcQi52FVMANEWwQJT0y5s27\nloULX4ur2LsMyTq/rsUc44MACgpKOOmko5SWXtim1nDrret5+mkvR2TNmmUMHry+ed++loOhdD/J\nmKrG9cRAFKW7SSXkM17IeKadesSBXINbORcOM2rUSa1cJz4nJAt57vpPZHF360MF+2LU1GSxbt3X\nyMj4DbGYX7MJajm1tZ9k4cLXWhlrLm6Xwl27tjFrlqcFdMSk1Nr5y8qa2tzekzkYqu30DgkFhzHm\nCuBta+0OY8wMJDX1NWCBtbaxJwaoKF1FKg7ceCETdHC7hQWXNf998smPtrjO+vW7qKqqRhbygwwe\n/DYNDeOJxZYB5wEPkZGxl2HDRjFkyA+prCygri4fSfYLkZGxj2h0mO+6rpbj5oXUs2HDPiZNOhw3\n1mpCobeJxWqIRGa1arJKhZafRTXFxY1tbu/JHAzVdnqHRJnj3wGuBW4wxnwMeSz6D0Svvg/4Zo+M\nUFG6iFSetr3Fv56qqv3AicAyjjvuPcaM+TNjxhwAGlrNU3Cvc8MNj7NmjWdaqq/3KsGKAzqHaHQu\nVVUhqqpiHHPMfLwquJVkZBwlM7OKxsY7gJEMHVpNY+PtjnCRZLtIJEZ9/YNkZv6AxsZipGBiHrFY\nASKwngKmdUoLKC29iPr63/LSSxnAQSZNGkZp6RUtPqveiKjqexnnA4NEGsf1wCRr7VFjzD3AKmvt\nUmNMCHi7Z4anKL2Du/hLQt9Nze8XFv6R0aMPOItoNZMmxSgtvbJV80h5eTACyl+CIyengezsRqdc\nu7wXCp2MCBSAN6mv9/qEX3bZb3n44a9QWRnhnHOeCRz36quDaWw8BRFKfj+Hpxn5NYSOfBYPP3xt\nwu299ZTf9zLOBwaJBEfMWnvU+ftCpCIu1tqY0wGwQziC51dIjela4CZr7Q7f9usQzaYB2GKtvbnD\nF1OUThK/MFVUlPHmm17+w5o1y3jppeVMmTKqhX29NROPECM7ey9DhtQTiXjbhw/fSU3NeMSPMgR/\nkyMRVLJIT5kyiJUrveOOHClHepovB4aRkbGFaNT92Ujp8tLSS7rxU+o9NH+kd0gkOBqNMXmIh+/j\nwFoAY0wx4nnrKDOAbGvtZGPMOcBPnPcwxhyDxCueYa2tM8Y8Zoy5wlr7ZCeupygdJn5h2rGjxKmc\nC64WEYmMZ+XK89m0aTGjRp1OcfEh5s6dyOHDB8nK+h7R6IkUFHzAmWfm8+qrDxGJHEMkMotIJMbo\n0XdTW1sEHCQWG4Q/asrf5CgalaitiooI9fUNZGY+SGPjQSBKQ0ODs/91QIzRo7cQDnsl16dMyeyw\nw7ivO581f6R3SCQ47gE2O/sstdaWG2OuARbSsgBiKpyP1FXAWvuy09/DpQ6YbK2t842vthPXUpRO\nEb8wzZr1BFu2xGsRIeBpwuG5hMOH2Lz5Kf7yl400NtYB3wLy2bcvxrBhj3LiicN9taxg374iotFs\n4BpgPUHTlhvBVU1t7X4qK2URX7PGXz5dMr4zM+czbNgIoIDTTsvnrLN+TXn5GMaM2cPhw1BS8t90\nJAdDnc9KayQqcvgnY8z/Asdaa99w3j6MmJae78Q1hyM6uEujMSbDWhu11rp5IhhjbgGGWmuf7cS1\nFKVL8TuKjxwpp6FhFPA54G/IYi75E42N8eVEQs2Ls998FY3mIKU+7kVqWMUXPQQIUV8/jzlzVrcS\nGlsHPElTUzaRyE1APuvWxZg+/VHWrr2YWbNWsGaNl4+Sag6GOp+V1kgYjmutDSPpru7rp7rgmlV4\nXkKADGtt1H3h+EBKgVORpshJUViY2/5OaYzOr29QWJjLU0+Js7yiIsLs2WvYufMFysvfYvfuK2mZ\nu+GVEykpqWHx4unMnr2cJ59s4OjRLNzQW+nRcTmunyIr6w0aGk7H6+cB4XA+JSXVcX6TwcCVxGJX\n4BdS4XA+hYW5hMP5cePJJRz2Pu/47e5xLiUlRwPXKympaXGv0uXedZT+Pr+O0Bud/DYimU9/Msac\nC2yJ2/4boMZaO6PFkQnozxUs+3uFzvSd36Dm7neVlROYM+dR1q//kKoqN7EvRmbm/5GbW8G55zay\nYMFnaGqSY+rqnmDlys8375eVtc3p0Cd+ik9/ei+vv76TcNhbtIuKKlmw4ELq6h5lx44hbN1qaWj4\nujOWoJAqKqpk//5qiooqiC+wWFTU2Px5x293j3NZsOCfqKvzfDwLFlwY2J6+9y45BsL8OkIoFuvZ\nnky+qKqPOW/dCExEvvWvIoV7XnC2xYCfWWtXtnPaWH+/uTq/volbx+nvf88ADpCZeYADByYgC/Eu\nhg8v5NJLh/Dtb3+MRYteazZXzZs3sTnru7i4innzzg68dqODvNpRVc1lQVxmzXqClSu/hLfoL3Kc\n896+lZUyPn8Oxv33e/3EKysjCa+RaN633rqecDifoqKKPuc07yrS+buZDIWFuaH292pJjwuObkIF\nRxqTzvOTXt3+Pts/RpLzliG+C3l/9Oi72bPn9ubXkpfRdm6En+3by7j66lUt+n93dNFvi1QiqOLn\nPX16/3Sap/N3Mxk6Kjh6w1SlKP2Gls7qYxEBEvR17NtXFHjt5mUkw9VXryIcbtn/u6tDUVOJoFKn\n+cAm+W+voiiAPJnPmrWCqVPXsW/fW3gNMsVXUFS0iLy8dwLvDxq0K26/g0lfr7LyBPyLtLzuelIR\nBsXFh/DPRzO2BxaqcShKigRLrk8JJPGJD+F6AObMEadySUkNBw8OZ926Zbi+j2i0MNBoKZGZKD//\nA2pqPAd2fv7upMeaivkplfIds2efyl//eid1dePIzt7B178+LekxKemPCg5FSZHgk3k+o0efwdq1\nF7fYzzXzFBbm8u67HzBnzno2bNhLJHIbVVUhp2yImIMSmYlWrJjOzJmLHB/HblasuKrFtdoiFfNT\nKuU7vvzl56mt/QFS3j3Gv/3bIjZvPjPpcSnpjQoORUmRVAvrHTzoObGlWk9Lc1AiM9G4ccVs3tyx\n3mmpmJ9S8Zn0lPlM6Zuo4FCUFEm1sN7NN6/xmbaCTZtcodNdVV6767ydMZ8p6Y+G46YBAyAkMC3n\n157/wN3+zDNw9Ggm4PoBlpKXN5IpUzID+RZdGVrr0l3n3bmzjJkzVxGJnEBenpjPxo0r7vR5+xrp\n+t1MFs3j6N83t79/edNyfu3lMrTM8ZCChLCcCRNyWvWLpBvpeu+SZQDMr0OCQ8NxFaWDtOc/aJnj\n0YAIj0s1fFVJa1RwKEoHaS+XIX57UdFuJkzIYfr01dpwSElr1DmuDGhefXULM2euac5HWLVqGhMm\nBMNKXV/F9u2DqKgoo6CghJNOOsK8eRNJ5CSPz3X4/e9bnru9a/TXGlDJ0tcbSQ1UVHAoA5qZM9cE\n8hGuuupO3n8/uLh7uRDLgbmEwyGnmVPi+kyp5Dp09Br9HW0k1TdRU5UyoKmrG4ffDyGvg3i+imD9\nqfbqM6WS69DRa/R3tCZW30QFhzKgyc7egd8PkZ29s8U+nq+imtZ8Gtu3lzFhwi8oLl7BhAk/Z+fO\nMkByHfz7J8p1aO8aAxWtidU3UVOVMqBZtWoaV13l+iF2smrVZS32cRP+xP+wyPE/HG32abRVvdYt\nFeLPdWiL9q4xUEk12VLpGTSPIw0YALHkaT2/4uIV1NR8qfl1Ts4jlJV5dvh0n18i+vPcYEDMT/M4\nFKU3SMUkpSj9ATVVKUon6Uz1WkVJR1RwKEoniUb9r7rO9Ks5DEpfRQWHonSStpzjnUVzGJS+ivo4\nFKWTdFdvCs1hUPoqKjgUpZO05Rx3e5N/6lOrmTXrCSorIymdV3MYlL6KmqoUpZO05RwP9iZP3dSk\nOQxKX0UFh6J0knHjinnuuesdR/bpLFz4KqWlIzptakqllaui9CQ9LjiMMSHgV8BZQC1wk7V2h2/7\nlcB8pHnB76y1S3t6jIqSKq05souLY93StlVRepve0DhmANnW2snGmHOAnzjvYYzJdF5PBGqAjcaY\nldba/b0wTkVJmta0i8cfl7Lr4XA+RUWVampS+g294Rw/H3gawFr7MvAJ37bTgG3W2iprbQPwInBB\nzw9RGSi4DuypU9d1yIHtIo7sSmAZsIp9+97CmMmsXPk2mzbtZ+XKtzDmjK4cuqL0Gr2hcQwHDvle\nNxpjMqy10Va2VQMjenJwysCiq3IlSksvYtOmxc35HOHwlcjXdxGec7xDZYEUpc/RG4KjCsj1vXaF\nhrvN70HMBZJ6BCwszG1/pzRG59c9hMP5+E1M4XB+h8ZSWJjL8cefSTjs7zFeEjg3lPTL+9gf5+Sn\nv8+vI/SG4NgIXAH8yRhzLrDFt+0d4BRjTB5wFDFT/SiZk/bzCpY6v26iqKgCTxuIUVRU2eGxxJ8L\nbNzrd/vdfdTvZnrTUaHYG4JjBXCJMWaj8/pGY8x1wFBr7VJjzLeBtcivbam1trwXxqgMELoyVyL+\nXCtX/hpP83gXWAzc3hXDVpReRftxpAED4alH55ee9Oe5wYCYn/bjUBRFUbofFRyKoihKSqjgUBRF\nUVJCBYeiKIqSEio4FEVRlJRQwaEoiqKkhAoORVEUJSVUcCiKoigpoYJDURRFSQkVHIqiKEpKqOBQ\nFEVRUkIFh6IoipISKjgURVGUlFDBoSiKoqSECg5FURQlJVRwKIqiKCmhgkNRFEVJCRUciqIoSkqo\n4FAURVFSQgWHoiiKkhIqOBRFUZSUUMGhKIqipIQKDkVRFCUlMnv6gsaYY4A/AKOAKuAGa+3BuH2+\nBVwLxICnrLULenqciqIoSuv0hsYxG3jDWnsB8Cgw37/RGDMOuM5ae661dhLwGWPMGb0wTkVRFKUV\nekNwnA887fy9Bvh03Pb3gUt9r7OA2h4Yl6IoipIE3WqqMsZ8GfgWYnICCAF7gEPO62pguP8Ya20T\nUOEc/yPgNWvte905TkVRFCV5ulVwWGsfAh7yv2eM+R8g13mZC0TijzPGZDvHHQJu7s4xKoqiKKnR\n485xYCMwDXjF+f+FVvZZBTxrrf1RkucMFRbmtr9XGqPzS2/68/z689yg/8+vI4RisVj7e3Uhxpgc\n4GFgDFAHfMFau8+JpNqGCLPHgL8jpq0YMNda+3KPDlRRFEVplR4XHIqiKEp6owmAiqIoSkqo4FAU\nRVFSQgWHoiiKkhIqOBRFUZSU6I1w3C7DGDMT+Ky19outbLsfOA9JMgSYbq2tjt+vL9PO/GYBXwEa\ngLuttX/p6fF1hCRrlaXdvTPGhIBfAWchlQ5ustbu8G2/Eimv0wD8zlq7tFcG2kGSmN83gZuAfc5b\nX7XWbuvxgXYCY8w5wD3W2gvj3k/re+eSYH4p37u0FRzO4jIV2NzGLhOBz1hrK3puVF1HovkZY44D\nbgHOBoYALxpj1lprG3p2lB3CrVX2A2PMtcgP8ptx+6TjvZsBZFtrJzs/0J8472GMyXReTwRqgI3G\nmJXW2v29NtrUaXN+DhOB6621/+iV0XUSY8x3geuBw3Hv94d71+b8HFK+d+lsqtqILEItcJ6OTgV+\nY4x50RhzY4+OrGtoc37Ap4AXrbWN1toqJP/lYz02ss6RsFZZGt+75nk5OUef8G07Ddhmra1yhPuL\nwAU9P8ROkWh+IIvPXGPMC8aY23p6cF3Ae8DMVt7vD/cO2p4fdODe9XmNI67elZsQeKO19r+NMVPa\nOGwo8HPkSSETWG+M2WStfbMnxpwKHZzfcLx6XyBPESO6daAdoCO1ykijexdH/D1pNMZkWGujrWyr\npg/er3ZIND+AZcADiPnxz8aYadbap3p6kB3FWrvCGFPcyqb+cO8SzQ86cO/6vOBord5VEhwFfm6t\nrQUwxjyH2Gb73OLTwflVEVxwW6351dt0sFZZ2ty7OKrw5gXgX1TT4n61Q6L5AfzM0X4xxvwF+DiQ\nNoIjAf3h3rVHyvcunU1ViShBbJEhY0wWoma/1stj6kr+DzjfGDPYGDMCGE/fX1hd3Fpl0HqtsnS9\nd83zMsacC2zxbXsHOMUYk2eMGYyYOl7q+SF2ijbnZ4wZDrxpjBnimBovAl7tlVF2nlDc6/5w7/wE\n5tfRe9fnNY5UcOtdWWufNMY8ArwM1AMPW2vf6d3RdZ64+f0csbeGgHnW2vreHV3SLAYeNsa8gFOr\nDPrFvVsBXGKM2ei8vtEYcx0w1Fq71BjzbWAtcr+WWmvLe2ugHaS9+c0FnkcirtZZa59u4zx9nRhA\nP7t3flqbX8r3TmtVKYqiKCnRX01ViqIoSjehgkNRFEVJCRUciqIoSkqo4FAURVFSQgWHoiiKkhIq\nOBRFUZSU6Fd5HMrAximp8C7wlvPWYOBDpIRLOMFx64HvWWv/luR1rgQmWmvvMsbcBTxjrd3YzmHu\nsZ8FbkN+eyHgUWvtfckcqyh9BRUcSn/jQ2vt2e4LY8xC4JfAv3TVBay1q4HVzsspwHPJHGeMKQLu\nAyZYayPGmCHABmPMVmvtk101PkXpblRwKP2dvwFXQnOpjPuBbOAA0nfA31NiEJLZ/lHgOMAiAmc0\nUhl2P5Jd+0fgnxGB8QlgqTHmX4C/WGuLnXNdANxmrXXLqwAci/zmhgERa+1RY8wNzjkxxnwaESwh\noAzJqj/ijPliIAr8wVpb6hTALEXMzW8C30AK1X0UGATca6193BhzJvAb571aRPva3rmPVBnoqI9D\n6bc4ta6uRfqVZCFVQG+21n4ceBBYHnfIZKDOWnseUtp9CF5drRLgi9baqc7rmLX2UeAV4N+d6r07\njDH/7Gy/Afid/+TW2jeAVc5+Lxtj7gEyrbU7nDpIf0D6IpwFvOGc42vACdbaM4BzgKuNMZc5pzwV\nuNBaeyNwB/CKtfaTiBZ0hzFmHFKd+D5r7aeAXwDnduCjVJQAKjiU/sbxxpjXjDH/wGuCNRdZ+Cus\nta8BWGv/BJxsjGmu+GqtfQFYbIy5GfgZcAqiHQDss9Z+0MY13cJxvwOuN8bkIBrCn+N3tNbeDBQj\n3fSKgZeMMTOAM4Hd1totzn53WGsfQIrO/d55rwbRdi72TmfdxjyfBr7mzPtvQA5wOvAk8IAxZinS\nwe6xRB+eoiSDmqqU/kbAx+FijBlLy8qnIcSE4+5zJfAD4KdIOfhjfcfUJHHt/wbuBj6LmK0CHRmN\nMdOAYdba/wIeRoo93gT8O3C7f3xO1dJcWj7chfB+t/4xDQL+1Vq72Tl+FHDQWttkjHkJuALptDgN\naTmsKB1GNQ6lvxEvHFwsMNIYMxHAGHMNUGat9fdW+DTwuLX2EaT/8gV4gqWt8zYCWdCsEaxBhMfv\nW9n3KLDQbajjlLE+HSkbb4FjjTHjnX3nAF8F1gH/ZozJcJzpXwTWt3Lu54CbnfOOQUxdHzHGLAfO\nsdYuQdr0fryNeShK0qjgUPobrZZ7dsrOX4uYbd5AFtlr4o5ZAnzBGPMq8Cek78K4ROdFnOaLHcc7\nwONAlbV2UytjeB74PvCkMeYd4G3kN7jAWlsH/CvwqDFmM9Ky9B7Esb0beB3pk/Bna+3KVsbxfSDH\nGLMFeBb4jrV2J7AQmOfM6UeIz0NROoWWVVeULsKJyrob2GOtvb+3x6Mo3YX6OBSl69iEhOxe1dsD\nUZTuRDUORVEUJSXUx6EoiqKkhAoORVEUJSVUcCiKoigpoYJDURRFSQkVHIqiKEpKqOBQFEVRUuL/\nAdjesy/F1rM8AAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x114a58610>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Plot scatter plot of polarity vs subjectivity scores\n",
    "plt.scatter(yelp.polarity, yelp.subjectivity)\n",
    "plt.xlabel(\"Polarity Scores\")\n",
    "plt.ylabel(\"Subjectivity Scores\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 88,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<matplotlib.axes._subplots.AxesSubplot at 0x117600290>"
      ]
     },
     "execution_count": 88,
     "metadata": {},
     "output_type": "execute_result"
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEcCAYAAADKlrO6AAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAAFwNJREFUeJzt3Xu03WV95/F3IHLtAaIeWrSX6RT9BhURUXAAEVI0RcXi\n6NSBSsslKHhZQjvaoQV7WdJmhkqpWkBuURSoWobxQrmKNSQqLVqVW76kC9Z0DcIQJJBIDBBy5o/f\n75Cds/fJOezL2Zfn/VorK2fv/du/37OfnPw++3l+z/P85k1MTCBJKtd2/S6AJKm/DAJJKpxBIEmF\nMwgkqXAGgSQVziCQpMLN73cBNLoi4k3Al4G7qb507ACclpk/6sK+lwFXZ+ZN07z+KmBBZt7W6bF6\nISL+Crg3M69oeO5PgYcy8+Iu7P8U4PLMfLbTfWn02SJQr30zMxdl5uHAnwKfmKPjvgt4xRwdaxD9\nMbB9vwuh4WCLQL02r+HnFwL/DyAi9gc+BWwCNgKnAC8Gvgi8HvivwGLgj4CvAD8Bfhm4PjPPntxh\nRMwHlgH/keqLzd8AK4ATgKci4vuZeUfD9mcDxwBrgF2As4AjgIOBXYGTgbcD7wGeAZZn5pmN39Yj\nIoCLMvOIiLgbuA14JfBT4Nj6fRcBe9dlOiszl0fEu4A/AR4BdgTubVFf/zki3gPsDHwE2AM4JTN/\npy7/CuDdmflw/fjFwJfqet4JOBV4HfBLwN9HxLuBz9Z1txfwtcz8eN2ielH9b3LM1H1k5o9blE0j\nyhaBem1RRNwaEd8BLgP+vn7+YuADmXkEcCHwN5n5Q+AS4ArgA8BJ9ba/Bvw+cGC9v/0b9v9+4JHM\nPAR4M1WLYyPwOeC8KSHwamBxZh5AdfL7pYb93JOZhwIvAN4NvKHe58si4m0tPtfklPxdgC9k5huB\nVVQn4iXAmroVdAxwQR1YnwQWZeZvARumqa/7M/M3631clJk3A6+KiN0j4hX1fh9u2P5A4FHgKOBD\nwK6ZeTnwEFWY/Qrw3cw8CjgIOK3hvd+sP3PTPqYpm0aUQaBem+waOhjYH/hSROwEvCQz76y3Wc6W\nbpzPAr8JXJmZkyfLH2XmE5m5GfhnINhyIt6nfj+Z+TPgHuA3pinLPvX7ycyNwPcbXsv674XA9+pj\nQdW6eOWU/TS2cp7JzJX1z9+ty7Yv8LaIuBW4hqqLZi/gscx8vN72O9OUcfKz3AP8Yv3cF4HjgBOp\nwrTR9fW+vgb8OTBZ7nn1n8eAAyPiC1StpR1afObp9qFCGATqtcaT5hqqE/gE8GBE7Fs/fzhwX/3z\nucD/BE6IiF+vn3tFROwUEdtTfau9u2G/9wCHAUTEGPAq4AGqk9nUPvK7qbqdiIgdqYJp0uTJbxVw\nUERsFxHz6n0nVSvjJfU2BzS87wUNn+MQ4C6qLp+rMnMR1bfsrwAPA7tHxIvqbV/fVFOVA+vy7Qv8\ne/3c54D/ArwR+Mcp2x9O1WW1GDgH+Mv6+Wfrz38CsDYzjwfOo2rBTP3M0+1DhTAI1GtH1F1DtwA3\nAGdk5lPA+4DPRMRy4MPAGRHxDuBlmbkUOJ3qm/ALgKepTqbfBa6tWxKTLYJLgBdFxG3ArcCfZeaj\nVN/2P1iPXAIgM+8Cro+I71F9U3+aqj9/Yso2X6b6hvw94IHM/CpVH/pb62/5r5nyGf+oPv5LqFo0\nFwP7RMQ/ASuB/5OZz9Sf86aIuKn+XK38ekR8E7iAqtuLzPwJsJ6qdTX12/qPgCUR8S2qAJ08ia8A\nrgNuAY6qy3IBcF9E7NX4mbexDxVinquPapBFxK9RDRM9uAv7Gqe60HphROxA9e19UWb+3w72+QDw\n8vpE3zMR8XXgI5l5fy+PozI5akgleRR4fUScSNUtckknIVCbYOvur66qr6esAG4xBNQrtggkqXBe\nI5BmEBGbI+KFz/M9l0TEovrni6cMeZUGil1D0syed7M5M09pePhmqglm0kCya0hFqUcRnQs8SDUb\neQPVEMuHgL+jGhG0mWqE05mZuTkiNlPNet5INfntZVQzctcDx2Xm6nrEzWNU8wguolri4tPAa4GP\nAvdTjQL6BvDSzFxflyepLmBPzqmQ5pxdQyrR/sC5mbkf1fIUX6Ra7uLRzNyXaomG/YD/NuV9R1GN\nyT84MxcCd1DNxJ30WGa+KjM/M/lEZp5FtTzGcZm5nGo45+8C1F1HjxoC6jeDQCX6UWZOzuxdRhUM\nxwKfAaiHgl5EdeJ/TmZeA3w+Ij4UEedTTcT6hYZNtrXS6eTIoguo1lWCai7Fhe1/DKk7DAKVaFPD\nz/Ma/jTajimTviLiNKolHp4ErgSunvK+n8104My8Bdilbg28kWrymtRXBoFKtH99vwKo+u1XUM0c\n/hA8t/zE+4Cp9zp4C7AsM5cBq4Gjmd1Sz5vYOlQuBC6lWk/p6XY/hNQtBoFK9DBwTkT8GHgHcDzV\nks97RsSdVEsurGLLUguTIyr+Gjg1In4A3Ey1jMXeU7ahxeP/TbXY3pH14yuoloX+bNc+kdQBRw2p\nKPWooU9n5qv7WIZjgfdmZqvlraU519E8gog4CFharynf+PzRwNlUC3oty8xLOzmONCrqYaZ7Ug0v\nlQZC2y2CiPgoVZP6Z40LgtU34LiXaqnen1Otvvi2zFzTeXElSd3WyTWCfwPe2eL5fYDVmbmuHoa3\ngnq9eEnS4Gk7CDLzWrYehjdpN+CJhsfrgd3bPY4kqbd6sdbQOqowmDQGPD7Nts/ZtOnZifnzZzMS\nT5LUppZLpncjCKbu+F5g74jYg2odl8Oo1nbZprVrp7uX99wbHx9jzZr1/S7GQLFOmlknzayTZoNU\nJ+PjYy2f70YQTMBzQ+J2zcxLI+IPqCbjzAMuzcyHunAcSVIPDMw8gjVr1g9GQRisBB8U1kkz66SZ\nddJskOpkfHysZdeQM4slqXAGgSQVziCQpMIZBJJUOINAkgpnEEhS4QwCSSqcQSBJhTMIJKlwBoEk\nFc4gkKTCGQSSVDiDQJIKZxBIUuEMAkkqnEEgSYUzCCSpcAaBJBXOIJCkwhkEklQ4g0CSCje/3wWQ\npGF12GEHsWrVvR3vZ+HCfVi+/PYulKg9BoEktWk2J+/x8THWrFk/B6Vpn11DklQ4g0CSCmcQSFLh\nDAJJKpxBIEk9dNWNq/pdhBkZBJLUQ1fflP0uwowMAkkqnEEgSYUzCCSpcAaBJBXOJSYkzcqorKsz\n1459S/S7CDMyCCTNymxO3jf/4EHe/NqXzkFphsdxixe61pCkcgzDUEk1MwgkqXAGgSQVziCQpMIZ\nBJLUQ641JKkowzBUcq4NwwV0g0BS1xy3eGG/i6A2GASSVDiDQJIKZxBIUuEMAknqoWG4gN7WWkMR\nMQ+4ANgP2Agsycz7G14/HVgCPFI/9f7MXN1hWSUNuKtuXOVaQ1MMw1pD7S46dwywY2YeHBEHAefV\nz006ADg+M/+10wJK/eBKm+25+qY0CIZQu0FwKHADQGbeHhGvm/L6AcCZEbEXcF1mLu2gjNKcm83J\n+6Slt3L5f180B6WReqvdawS7AU80PN4UEY37uho4FTgCODQi3trmcSRJPdZui2AdMNbweLvM3Nzw\n+G8zcx1ARFwH7A/847Z2uGDBLsyfv32bxem+8fGxmTcqjHWytWPfEtZJC9ZJs0Gvk3aDYCXwduAf\nIuINwJ2TL0TEbsBdEbEQ+DmwCLhsph2uXbuhzaJ03/j42MBf3Jlr1kmzYbgI2A/WydYG6WY90wVS\nu11D1wJPRcRK4JPAGRFxbEQsqVsCZwL/BHwbuCszb2jzOJKGyDAMlZxrw7DWUFstgsycAE6b8vR9\nDa9fCVzZQbkkDSFbScPJCWWSVDhvXq+ujJkvbby8NEoMAjlmvk3OotWoMAikNjmLdvR9+PzlPLlx\nU8f7OWnprR29f9ed5vPp0w/ruBzTMQgkdc2otZKe3Lip45ZwN4ZedxokM/FisaSuGYahkmpmEGhW\nHB8ujS6DQLPivWil0WUQSG2ylaRRUdzFYsfMq1ucRatRUVwQzOYEPkiLRElzpZShkmpWXBDMhuPD\nVaJShkqqmdcINCtX3biq30WQ1CMGgWbF8eHS6DIIpDbZStKoMAikNtlK0qjwYnELjg+XBHDyv3+N\n+5Zc0dE+7pt5k5nLscMeVHf97Q2DoAXHh0sCuOxX3zEQI6mWLr2VQzraw7bZNaRZsZUkjS6DQLPi\nWkPS6LJrSEVyFq20hUGgIjmLVtrCIGhh1O6yJM1GKSNk1MwgaMG1hlSiUkbIqJkXizUrzqKVRpdB\noFlxFq00ugwCSSqc1wgkaRsGYWTXrjv19lRtELTgLFpJQMcXz6EKkm7sp5fsGmrBWbSSSmKLYMR1\nawYtjNYsWsfMS1sYBCOuGzNoYfRm0TpmvrVB+DfqdX+4mlnjkoBy+sPVzGsEktRDwzD4xCBowVm0\nkrplGAafGAQtOItWUkkMAkkqnEEgqWuGoT9czQwCSV0zDP3haubwURXLMfOaC8Nwo6t5ExMT/S4D\nAGvWrO+4IN2cRdupQZlFu/IDpzP+9OP9LgYAa3bYg0MuOL/fxegax8w368Yku1EzSL8n4+Nj81o9\nP1JfR5xF26wbM2hhNGfRSqp4jUCSCmcQSOoaJ2MOJ4NAUtc4GXM4GQRSmxwzr9kYht+Tti4WR8Q8\n4AJgP2AjsCQz7294/WjgbOAZYFlmXtqFskoD5bjFCx0hoxkNw+9Juy2CY4AdM/Ng4EzgvMkXImJ+\n/fhI4HDgfREx3mE5JUk90m4QHArcAJCZtwOva3htH2B1Zq7LzGeAFUD/B9RLklpqdx7BbsATDY83\nRcR2mbm5xWvrgd3bPM7z0o3bD8Lo3YJwUOY0OIt29A1Df7iatfs/cx0w1vB4MgQmX9ut4bUxYMap\nrQsW7ML8+du3WZzKZb/6Dr7+yd/uaB/dcvQffpVjxsdm3rDHulUfR//hVwembgfJ+AD8Gw8S1xpq\nbdB/T9oNgpXA24F/iIg3AHc2vHYvsHdE7AFsoOoWOnemHa5du6HNomytGxdlujVNftAvED1fo/Z5\nOnXzDx4c+DVk5ppLTDQbpN+T6QKp3WsE1wJPRcRK4JPAGRFxbEQsycxNwB8AN1EFxqWZ+VCbx5EG\nlmPmNRvD8HvSVosgMyeA06Y8fV/D69cB13VQLknSHHFCmSQVziDQrDgaRLPhWkPDySDQrDgaRLMx\nDP3hamYQSG2ylaTZGIbfE4NAapOtJM3GMPyeGASSVDiDQJIKN3KLv7iuTm9cdeOqgZkdqcE1DP3h\najZSZ6tu3KQdqjDp1r5GxdU3pUFQuMMOO4hVq+7teD8LF+7D8uW3d6FE/TcqdTJSQSDNpdJaSbM5\nUZW21tCo1InXCKQ2OWZeo8IWgdTCbJv8e5637df73eSXZsMgkFoYlSa/NBt2DbXgyIdm1ok0ugyC\nFoZhJuBcs06k0WUQSFLhDAJJKpxBIEmFMwgkqXAGQQveZamZdSKNLoOgBWeMNrNOpNFlEEhS4QwC\nSSqcS0yoK+vquKaONLwMArmujlQ4u4ZacF0dSSUxCFpwXR1JJTEIJKlwBoEkFc4gkKTCGQSSVDiD\noAXX1ZFUEoOgBdfVkVQSg0CSCmcQSFLhDAJJKpxBIEmFMwhacK0hSSUxCFpwrSFJJSluGerZrr2/\nLa69L2mUFBcErr0vSVuza0iSCmcQSFLhDAJJKpxBIEmFMwgkqXBtjRqKiJ2ALwJ7AuuA38/Mn07Z\n5nzgEGBy+M1vZ6ZDcSRpwLQ7fPQ04MeZ+RcR8R7gbOD0KdscACzOzMc6KaAkqbfa7Ro6FLih/vl6\n4MjGFyNiHvAy4OKIWBERJ7ZfRElSL83YIoiIk4AzgIn6qXnAw8AT9eP1wG5T3rYr8CngvPoY34qI\nf8nMu7pRaElS98wYBJl5OXB543MRcQ0wVj8cAx6f8rYNwKcyc2O9/a3AfsC0QbBgwS7Mn7/97Eve\nY+PjYzNvVBjrpJl10sw6aTboddLuNYKVwFuBO+q/b5vy+suBL0XEa+pjHAp8bls7XLt2Q5tF6T6X\nmGhmnTSzTppZJ80GqU6mC6R2g+BC4PMRcRvwFHAcQEScAazOzG9ExBXA7cDTwOczs7OV3iRJPTFv\nYmJi5q3mwJo16wejIAxWgg8K66SZddLMOmk2SHUyPj42r9XzTiiTpMIZBJJUOINAkgpnEEhS4QwC\nSSqcQSBJhTMIJKlwBoEkFc4gkKTCGQSSVDiDQJIKZxBIUuEMAkkqnEEgSYUzCCSpcAaBJBXOIJCk\nwhkEklQ4g0CSCmcQSFLhDAJJKpxBIEmFMwgkqXAGgSQVziCQpMIZBJJUOINAkgpnEEhS4QwCSSqc\nQSBJhTMIJKlwBoEkFc4gkKTCGQSSVDiDQJIKZxBIUuEMAkkqnEEgSYUzCCSpcAaBJBXOIJCkwhkE\nklQ4g0CSCmcQSFLhDAJJKpxBIEmFm9/JmyPincC7M/N3W7x2CvA+4BngnMy8rpNjSZJ6o+0WQUSc\nD5wDzGvx2i8CHwb+E/BbwF9FxAvaPZYkqXc66RpaCZw2zWsHAisyc1NmrgNWA6/u4FiSpB6ZsWso\nIk4CzgAmqL79TwAnZuZXIuJN07xtN+CJhsc/A3bvsKySpB6YMQgy83Lg8ue533VUYTBpDHh8W28Y\nHx9r6mLqp/HxsX4XYeBYJ82sk2bWSbNBr5OOLhZvwz8Dn4iIHYCdgYXAXT06liSpA10Ngog4A1id\nmd+IiE8BK6i6k/44M5/u5rEkSd0xb2Jiot9lkCT1kRPKJKlwBoEkFc4gkKTC9WrU0NCKiIOApZl5\nRL/L0m8RMZ9q6PB/AHagWirk630t1ACIiO2AS4AANgOnZuY9/S1V/0XEnsAdwJGZeV+/yzMIIuL7\nbJlT9UBmntzP8kzHIGgQER8FjqeaACd4L/BoZv5eRCwAfggUHwTA0cBEZh5aT6r8S+CYPpepr+ov\nDRcBG/pdlkERETsCZOaifpdlJnYNbe3fgHf2uxAD5MvA2fXP21EtIFi8zPwq1YKKULWW1vavNAPj\nr4ELgZ/0uyADZD9g14i4MSJuqXsbBpJB0CAzrwU29bscgyIzN2TmkxExBnwF+JN+l2lQZObmiPgc\n8LfAlX0uTl9FxAnAI5l5My0WoSzYBuDczFxMtS7blXW34sAZyEJpcETErwC3Ap/PzC/1uzyDJDNP\nAF4OXBoRO/e5OP10IvDmiPgW8Brgivp6Qenuo/6SkJmrgZ8Ce/W1RNPwGkFrfqvhueXEbwQ+mJnf\n6nd5BkVEvBf45cxcCmwEnqW6aFykzHxu8ck6DN6fmY/0sUiD4iRgX+CDEfESqjXXHupvkVozCFpz\nunXlTGAP4OyI+DhVvRyVmU/1t1h997+AZRHxbar/Qx+xTp7j/50tLqP6PbmN6ovCSZk5kF8YXGJC\nkgrnNQJJKpxBIEmFMwgkqXAGgSQVziCQpMIZBJJUOINA2oaI+LOIOKTf5ZB6ySCQtu1NwPb9LoTU\nS04ok2oR8VKqtWF2oZoJeh3wMaplAd4JvBj4BLAzsAD4WGZeExHLgBcBv1FvfzhwJNXSE1/LzL+Y\n208iPT+2CKQtTga+npkHUp3QnwT+BTg5M+8GPlj//DpgCfDxhvc+mpmvBO6kWoZjf+AQYO+I2GEu\nP4T0fLnWkLTFLcA1EfFa4BvA31HdhGZyEcLjgbdHxO8AbwB+oeG9t9d/PwhsiIgV9T7Oysyn56Lw\nUrtsEUi1zPwO8ArgBuA9VHdja+w7XQG8nup2jOew9Sq1P6/38SxVSJwFvBD4XkTs3fPCSx0wCKRa\nRPwP4Pcy8wvAh4HXUt2oaH59q869gY9n5g3AYlpcRI6I1wDfBpZn5seAe6jubSwNLINA2uLTwLsi\n4l+plpo+lep+DBdRncwvBe6pb0j+YmDn+oY0z7UaMvOHwHeAuyPiDuAB4Po5/RTS8+SoIUkqnC0C\nSSqcQSBJhTMIJKlwBoEkFc4gkKTCGQSSVDiDQJIKZxBIUuH+PxwWCg3WTVdCAAAAAElFTkSuQmCC\n",
      "text/plain": [
       "<matplotlib.figure.Figure at 0x11673c610>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "#Plot boxplots of the polarity by yelp stars\n",
    "yelp.boxplot(column='polarity', by='stars')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Part 9: Calculating \"spaminess\" of a token"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 89,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>label</th>\n",
       "      <th>message</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>ham</td>\n",
       "      <td>Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>ham</td>\n",
       "      <td>Ok lar... Joking wif u oni...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>spam</td>\n",
       "      <td>Free entry in 2 a wkly comp to win FA Cup final tkts 21st May 2005. Text FA to 87121 to receive entry question(std txt rate)T&amp;C's apply 08452810075over18's</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>ham</td>\n",
       "      <td>U dun say so early hor... U c already then say...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>ham</td>\n",
       "      <td>Nah I don't think he goes to usf, he lives around here though</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  label  \\\n",
       "0   ham   \n",
       "1   ham   \n",
       "2  spam   \n",
       "3   ham   \n",
       "4   ham   \n",
       "\n",
       "                                                                                                                                                       message  \n",
       "0                                              Go until jurong point, crazy.. Available only in bugis n great world la e buffet... Cine there got amore wat...  \n",
       "1                                                                                                                                Ok lar... Joking wif u oni...  \n",
       "2  Free entry in 2 a wkly comp to win FA Cup final tkts 21st May 2005. Text FA to 87121 to receive entry question(std txt rate)T&C's apply 08452810075over18's  \n",
       "3                                                                                                            U dun say so early hor... U c already then say...  \n",
       "4                                                                                                Nah I don't think he goes to usf, he lives around here though  "
      ]
     },
     "execution_count": 89,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Load in ham or spam text dataset\n",
    "df = pd.read_table(\"sms.tsv\",encoding=\"utf-8\", names= [\"label\", \"message\"])\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 90,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "ham     0.865937\n",
       "spam    0.134063\n",
       "Name: label, dtype: float64"
      ]
     },
     "execution_count": 90,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Look at null accuracy\n",
    "df.label.value_counts(normalize=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 91,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "0.99353912419239054"
      ]
     },
     "execution_count": 91,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "X = df.message\n",
    "y = df.label\n",
    "vect =CountVectorizer()\n",
    "Xdtm = vect.fit_transform(X)\n",
    "nb = MultinomialNB()\n",
    "nb.fit(Xdtm,y)\n",
    "nb.score(Xdtm,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 92,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "8713"
      ]
     },
     "execution_count": 92,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "tokens = vect.get_feature_names()\n",
    "len(tokens)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 93,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[u'00', u'000', u'000pes', u'008704050406', u'0089', u'0121', u'01223585236', u'01223585334', u'0125698789', u'02', u'0207', u'02072069400', u'02073162414', u'02085076972', u'021', u'03', u'04', u'0430', u'05', u'050703', u'0578', u'06', u'07', u'07008009200', u'07046744435', u'07090201529', u'07090298926', u'07099833605', u'07123456789', u'0721072', u'07732584351', u'07734396839', u'07742676969', u'07753741225', u'0776xxxxxxx', u'07781482378', u'07786200117', u'077xxx', u'078', u'07801543489', u'07808', u'07808247860', u'07808726822', u'07815296484', u'07821230901', u'078498', u'07880867867', u'0789xxxxxxx', u'07946746291', u'0796xxxxxx']\n"
     ]
    }
   ],
   "source": [
    "#Print first 50 features\n",
    "print vect.get_feature_names()[:50]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 94,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "[u'fifteen', u'fifth', u'fifty', u'fight', u'fighting', u'fightng', u'fights', u'figure', u'figures', u'figuring', u'file', u'files', u'fill', u'filled', u'filling', u'fills', u'film', u'films', u'filth', u'filthy', u'filthyguys', u'final', u'finalise', u'finally', u'finance', u'financial', u'find', u'finding', u'finds', u'fine', u'finest', u'fingers', u'finish', u'finishd', u'finished', u'finishes', u'finishing', u'fink', u'finn', u'fire', u'fired', u'firefox', u'fireplace', u'fires', u'firmware', u'firsg', u'first', u'fish', u'fishhead', u'fishrman']\n"
     ]
    }
   ],
   "source": [
    "#Print random slice of features\n",
    "print vect.get_feature_names()[3200:3250]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 95,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([[  0.,   0.,   1., ...,   1.,   0.,   1.],\n",
       "       [ 10.,  29.,   0., ...,   0.,   1.,   0.]])"
      ]
     },
     "execution_count": 95,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#How many times does a word appear in each class\n",
    "nb.feature_count_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 96,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "(2, 8713)"
      ]
     },
     "execution_count": 96,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "nb.feature_count_.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 97,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 0.,  0.,  1., ...,  1.,  0.,  1.])"
      ]
     },
     "execution_count": 97,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "ham_token_count = nb.feature_count_[0,:]\n",
    "ham_token_count"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 98,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 10.,  29.,   0., ...,   0.,   1.,   0.])"
      ]
     },
     "execution_count": 98,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "spam_token_count = nb.feature_count_[1, :]\n",
    "spam_token_count"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 99,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ham</th>\n",
       "      <th>spam</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>token</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>missions</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>residency</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>reset</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>chat80155</th>\n",
       "      <td>0.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>tight</th>\n",
       "      <td>2.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>another</th>\n",
       "      <td>35.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>gentleman</th>\n",
       "      <td>3.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>apply</th>\n",
       "      <td>2.0</td>\n",
       "      <td>30.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4the</th>\n",
       "      <td>1.0</td>\n",
       "      <td>0.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>virgin</th>\n",
       "      <td>1.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            ham  spam\n",
       "token                \n",
       "missions    1.0   0.0\n",
       "residency   1.0   0.0\n",
       "reset       1.0   0.0\n",
       "chat80155   0.0   1.0\n",
       "tight       2.0   0.0\n",
       "another    35.0   2.0\n",
       "gentleman   3.0   0.0\n",
       "apply       2.0  30.0\n",
       "4the        1.0   0.0\n",
       "virgin      1.0   1.0"
      ]
     },
     "execution_count": 99,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# create a DataFrame of tokens with their separate ham and spam counts\n",
    "df_tokens = pd.DataFrame({'token':tokens, 'ham':ham_token_count, 'spam':spam_token_count}).set_index('token')\n",
    "df_tokens.sample(10, random_state=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 100,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ham</th>\n",
       "      <th>spam</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>token</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>missions</th>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>residency</th>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>reset</th>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>chat80155</th>\n",
       "      <td>1.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>tight</th>\n",
       "      <td>3.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>another</th>\n",
       "      <td>36.0</td>\n",
       "      <td>3.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>gentleman</th>\n",
       "      <td>4.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>apply</th>\n",
       "      <td>3.0</td>\n",
       "      <td>31.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4the</th>\n",
       "      <td>2.0</td>\n",
       "      <td>1.0</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>virgin</th>\n",
       "      <td>2.0</td>\n",
       "      <td>2.0</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            ham  spam\n",
       "token                \n",
       "missions    2.0   1.0\n",
       "residency   2.0   1.0\n",
       "reset       2.0   1.0\n",
       "chat80155   1.0   2.0\n",
       "tight       3.0   1.0\n",
       "another    36.0   3.0\n",
       "gentleman   4.0   1.0\n",
       "apply       3.0  31.0\n",
       "4the        2.0   1.0\n",
       "virgin      2.0   2.0"
      ]
     },
     "execution_count": 100,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# add 1 to ham and spam counts to avoid dividing by 0\n",
    "df_tokens['ham'] = df_tokens.ham + 1\n",
    "df_tokens['spam'] = df_tokens.spam + 1\n",
    "df_tokens.sample(10, random_state=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 101,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "array([ 4825.,   747.])"
      ]
     },
     "execution_count": 101,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Naive Bayes counts the number of observations in each class\n",
    "nb.class_count_"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 102,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ham</th>\n",
       "      <th>spam</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>token</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>missions</th>\n",
       "      <td>0.000415</td>\n",
       "      <td>0.001339</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>residency</th>\n",
       "      <td>0.000415</td>\n",
       "      <td>0.001339</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>reset</th>\n",
       "      <td>0.000415</td>\n",
       "      <td>0.001339</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>chat80155</th>\n",
       "      <td>0.000207</td>\n",
       "      <td>0.002677</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>tight</th>\n",
       "      <td>0.000622</td>\n",
       "      <td>0.001339</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>another</th>\n",
       "      <td>0.007461</td>\n",
       "      <td>0.004016</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>gentleman</th>\n",
       "      <td>0.000829</td>\n",
       "      <td>0.001339</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>apply</th>\n",
       "      <td>0.000622</td>\n",
       "      <td>0.041499</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4the</th>\n",
       "      <td>0.000415</td>\n",
       "      <td>0.001339</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>virgin</th>\n",
       "      <td>0.000415</td>\n",
       "      <td>0.002677</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                ham      spam\n",
       "token                        \n",
       "missions   0.000415  0.001339\n",
       "residency  0.000415  0.001339\n",
       "reset      0.000415  0.001339\n",
       "chat80155  0.000207  0.002677\n",
       "tight      0.000622  0.001339\n",
       "another    0.007461  0.004016\n",
       "gentleman  0.000829  0.001339\n",
       "apply      0.000622  0.041499\n",
       "4the       0.000415  0.001339\n",
       "virgin     0.000415  0.002677"
      ]
     },
     "execution_count": 102,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# convert the ham and spam counts into frequencies\n",
    "df_tokens['ham'] = df_tokens.ham / nb.class_count_[0]\n",
    "df_tokens['spam'] = df_tokens.spam / nb.class_count_[1]\n",
    "df_tokens.sample(10, random_state=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 103,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ham</th>\n",
       "      <th>spam</th>\n",
       "      <th>spam_ratio</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>token</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>missions</th>\n",
       "      <td>0.000415</td>\n",
       "      <td>0.001339</td>\n",
       "      <td>3.229585</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>residency</th>\n",
       "      <td>0.000415</td>\n",
       "      <td>0.001339</td>\n",
       "      <td>3.229585</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>reset</th>\n",
       "      <td>0.000415</td>\n",
       "      <td>0.001339</td>\n",
       "      <td>3.229585</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>chat80155</th>\n",
       "      <td>0.000207</td>\n",
       "      <td>0.002677</td>\n",
       "      <td>12.918340</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>tight</th>\n",
       "      <td>0.000622</td>\n",
       "      <td>0.001339</td>\n",
       "      <td>2.153057</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>another</th>\n",
       "      <td>0.007461</td>\n",
       "      <td>0.004016</td>\n",
       "      <td>0.538264</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>gentleman</th>\n",
       "      <td>0.000829</td>\n",
       "      <td>0.001339</td>\n",
       "      <td>1.614793</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>apply</th>\n",
       "      <td>0.000622</td>\n",
       "      <td>0.041499</td>\n",
       "      <td>66.744757</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4the</th>\n",
       "      <td>0.000415</td>\n",
       "      <td>0.001339</td>\n",
       "      <td>3.229585</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>virgin</th>\n",
       "      <td>0.000415</td>\n",
       "      <td>0.002677</td>\n",
       "      <td>6.459170</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                ham      spam  spam_ratio\n",
       "token                                    \n",
       "missions   0.000415  0.001339    3.229585\n",
       "residency  0.000415  0.001339    3.229585\n",
       "reset      0.000415  0.001339    3.229585\n",
       "chat80155  0.000207  0.002677   12.918340\n",
       "tight      0.000622  0.001339    2.153057\n",
       "another    0.007461  0.004016    0.538264\n",
       "gentleman  0.000829  0.001339    1.614793\n",
       "apply      0.000622  0.041499   66.744757\n",
       "4the       0.000415  0.001339    3.229585\n",
       "virgin     0.000415  0.002677    6.459170"
      ]
     },
     "execution_count": 103,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# calculate the ratio of spam-to-ham for each token\n",
    "df_tokens['spam_ratio'] = df_tokens.spam / df_tokens.ham\n",
    "df_tokens.sample(10, random_state=3)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 105,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>ham</th>\n",
       "      <th>spam</th>\n",
       "      <th>spam_ratio</th>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>token</th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "      <th></th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>gt</th>\n",
       "      <td>0.066114</td>\n",
       "      <td>0.001339</td>\n",
       "      <td>0.020248</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lt</th>\n",
       "      <td>0.065699</td>\n",
       "      <td>0.001339</td>\n",
       "      <td>0.020376</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>he</th>\n",
       "      <td>0.048083</td>\n",
       "      <td>0.001339</td>\n",
       "      <td>0.027841</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>she</th>\n",
       "      <td>0.034819</td>\n",
       "      <td>0.001339</td>\n",
       "      <td>0.038447</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>lor</th>\n",
       "      <td>0.033782</td>\n",
       "      <td>0.001339</td>\n",
       "      <td>0.039627</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>da</th>\n",
       "      <td>0.031295</td>\n",
       "      <td>0.001339</td>\n",
       "      <td>0.042776</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>later</th>\n",
       "      <td>0.028187</td>\n",
       "      <td>0.001339</td>\n",
       "      <td>0.047494</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>ask</th>\n",
       "      <td>0.018653</td>\n",
       "      <td>0.001339</td>\n",
       "      <td>0.071769</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>said</th>\n",
       "      <td>0.018653</td>\n",
       "      <td>0.001339</td>\n",
       "      <td>0.071769</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>doing</th>\n",
       "      <td>0.018446</td>\n",
       "      <td>0.001339</td>\n",
       "      <td>0.072575</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "            ham      spam  spam_ratio\n",
       "token                                \n",
       "gt     0.066114  0.001339    0.020248\n",
       "lt     0.065699  0.001339    0.020376\n",
       "he     0.048083  0.001339    0.027841\n",
       "she    0.034819  0.001339    0.038447\n",
       "lor    0.033782  0.001339    0.039627\n",
       "da     0.031295  0.001339    0.042776\n",
       "later  0.028187  0.001339    0.047494\n",
       "ask    0.018653  0.001339    0.071769\n",
       "said   0.018653  0.001339    0.071769\n",
       "doing  0.018446  0.001339    0.072575"
      ]
     },
     "execution_count": 105,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# examine the DataFrame sorted by spam_ratio\n",
    "df_tokens.sort_values('spam_ratio', ascending=True).head(10)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 109,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1.6147925033467201"
      ]
     },
     "execution_count": 109,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Try looking up scores of different words\n",
    "word = \"table\"\n",
    "df_tokens.loc[word, 'spam_ratio']"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "- NLP is a gigantic field\n",
    "- Understanding the basics broadens the types of data you can work with\n",
    "- Simple techniques go a long way\n",
    "- Use scikit-learn for NLP whenever possible"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "ls"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Lab time\n",
    "- There are three other datasets pitchfork album reviews, fake/real news, and political lean.\n",
    "- Pick one of those three datasets and try to build a model that differentiate between good/bad review, real/fake news, or liberal/conservative leaning. Make sure to examine the false positives and the false negatives texts. Use the \"spamminess\" technique on the corpus as well. \n",
    "- Use both count and tfidf vectorizers. Use textblob to determine sentiment and polarity.\n",
    "- I've included some bonus material if you want to explore. \n",
    "    \n",
    "    -How to summarize a text\n",
    "    \n",
    "    -How to use gridsearch to find the optimal parameters for countvectorizer.\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Bonus: Using TF-IDF to Summarize a Yelp Review\n",
    "\n",
    "Reddit's autotldr uses the [SMMRY](http://smmry.com/about) algorithm, which is based on TF-IDF!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 125,
   "metadata": {},
   "outputs": [],
   "source": [
    "tfidf = TfidfVectorizer()\n",
    "dtm = tfidf.fit_transform(yelp.text)\n",
    "features = tfidf.get_feature_names()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 126,
   "metadata": {},
   "outputs": [],
   "source": [
    "def summarize():\n",
    "    # choose a random review that is at least 300 characters\n",
    "    review_length = 0\n",
    "    while review_length < 300:\n",
    "        review_id = np.random.randint(0, len(yelp))\n",
    "        review_text = yelp.text[review_id]\n",
    "        review_length = len(review_text)\n",
    "    # create a dictionary of words and their TF-IDF scores\n",
    "    word_scores = {}\n",
    "    for word in TextBlob(review_text).words:\n",
    "        word = word.lower()\n",
    "        if word in features:\n",
    "            word_scores[word] = dtm[review_id, features.index(word)]\n",
    "    \n",
    "    # print words with the top 5 TF-IDF scores\n",
    "    print 'TOP SCORING WORDS:'\n",
    "    top_scores = sorted(word_scores.items(), key=lambda x: x[1], reverse=True)[:5]\n",
    "    for word, score in top_scores:\n",
    "        print word\n",
    "    \n",
    "    # print 5 random words\n",
    "    print '\\n' + 'RANDOM WORDS:'\n",
    "    random_words = np.random.choice(word_scores.keys(), size=5, replace=False)\n",
    "    for word in random_words:\n",
    "        print word\n",
    "    \n",
    "    # print the review\n",
    "    print '\\n' + review_text"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 127,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "TOP SCORING WORDS:\n",
      "very\n",
      "dainty\n",
      "always\n",
      "are\n",
      "pile\n",
      "\n",
      "RANDOM WORDS:\n",
      "share\n",
      "do\n",
      "nice\n",
      "cuisine\n",
      "needs\n",
      "\n",
      "PHENOMENAL place for Chinese cuisine. The portions are HUGE! My fiance and I always only order one thing and share it, and still always have leftovers. The menu prices are very reasonable. Including tip, we usually don't spend more than $15 for the both of us for dinner. They are very personable and attentive to your needs. I love that it is a very dainty, and has a very laid back atmosphere. The lemon chicken is our favorite!! :) They are also very kid friendly. We have an 18 month old son who always leaves a nice pile of rice on the floor every time we leave lol, but they love him! Highly recommend!\n"
     ]
    }
   ],
   "source": [
    "summarize()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": []
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# df[\"text\"] = df.text.str.decode(\"utf-8\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "# df.sample(frac=.5)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Gridsearch/pipelining and vectorization"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 111,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "/Users/georgemcintire/anaconda/lib/python2.7/site-packages/sklearn/grid_search.py:43: DeprecationWarning: This module was deprecated in version 0.18 in favor of the model_selection module into which all the refactored classes and functions are moved. This module will be removed in 0.20.\n",
      "  DeprecationWarning)\n"
     ]
    }
   ],
   "source": [
    "from sklearn.grid_search import GridSearchCV"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 112,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#make a pipeline \n",
    "from sklearn.pipeline import make_pipeline\n",
    "pipe = make_pipeline(CountVectorizer(), MultinomialNB())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 113,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "pipe2 = make_pipeline(TfidfVectorizer(), MultinomialNB())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#pipe steps\n",
    "pipe.steps"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "#Set range of parameters\n",
    "param_grid = {}\n",
    "param_grid[\"countvectorizer__max_features\"] = [1000,5000,10000]\n",
    "param_grid[\"countvectorizer__ngram_range\"] = [(1,1), (1,2), (2,2)]\n",
    "param_grid[\"countvectorizer__lowercase\"] = [True, False]\n",
    "# param_grid[\"countvectorizer__analyzer\"] = [\"word\", word_tokenize_lemma]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 121,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "param_grid_2 = {}\n",
    "param_grid_2[\"tfidfvectorizer__max_features\"] = [1000,5000,10000]\n",
    "param_grid_2[\"tfidfvectorizer__ngram_range\"] = [(1,1), (1,2), (2,2)]\n",
    "param_grid_2[\"tfidfvectorizer__lowercase\"] = [True, False]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from sklearn.grid_search import GridSearchCV\n",
    "grid = GridSearchCV(pipe, param_grid, cv=5, scoring='accuracy')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 128,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "from time import time "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#This will take a while\n",
    "t = time()\n",
    "grid.fit(X,y)\n",
    "print time()-t"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "#Look at the best parameters and the best scores\n",
    "print(grid.best_params_)\n",
    "print(grid.best_score_)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 122,
   "metadata": {},
   "outputs": [],
   "source": [
    "grid2 = GridSearchCV(pipe2, param_grid_2, cv=5,scoring=\"accuracy\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 123,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GridSearchCV(cv=5, error_score='raise',\n",
       "       estimator=Pipeline(steps=[('tfidfvectorizer', TfidfVectorizer(analyzer=u'word', binary=False, decode_error=u'strict',\n",
       "        dtype=<type 'numpy.int64'>, encoding=u'utf-8', input=u'content',\n",
       "        lowercase=True, max_df=1.0, max_features=None, min_df=1,\n",
       "        ngram_range=(1, 1), norm=u'l2', preprocessor=None, smoo...   vocabulary=None)), ('multinomialnb', MultinomialNB(alpha=1.0, class_prior=None, fit_prior=True))]),\n",
       "       fit_params={}, iid=True, n_jobs=1,\n",
       "       param_grid={'tfidfvectorizer__ngram_range': [(1, 1), (1, 2), (2, 2)], 'tfidfvectorizer__lowercase': [True, False], 'tfidfvectorizer__max_features': [1000, 5000, 10000]},\n",
       "       pre_dispatch='2*n_jobs', refit=True, scoring='accuracy', verbose=0)"
      ]
     },
     "execution_count": 123,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#Helpful for understanding how to create your param grid.\n",
    "grid2.fit(X,y)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "anaconda-cloud": {},
  "kernelspec": {
   "display_name": "Python 2",
   "language": "python",
   "name": "python2"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 2
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython2",
   "version": "2.7.12"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}