Rapport.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Projet FRI\n",
    "Sylvestre Prabakaran\n",
    "\n",
    "Benoît Laurès\n",
    "\n",
    "Benoît Matet"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Librairies et outils de traitement de texte"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Modifier les paths suivants si nécessaire. Les données sont supposées être dans un dossier `data` puis dans leurs dossiers respectifs (`data/CACM` et `data/pa1-data`)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import matplotlib.pyplot as plt\n",
    "from time import time\n",
    "%matplotlib inline\n",
    "\n",
    "# Change path if necessary\n",
    "path_to_cacm_file = 'data/CACM/cacm.all'\n",
    "path_to_cs276_dir = 'data/pa1-data'\n",
    "path_to_qrels = 'data/CACM/qrels.text'\n",
    "path_to_query = 'data/CACM/query.text'\n",
    "path_to_common_words = 'data/CACM/common_words'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "[nltk_data] Downloading package punkt to\n",
      "[nltk_data]     C:\\Users\\Benoît\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Package punkt is already up-to-date!\n",
      "[nltk_data] Downloading package stopwords to\n",
      "[nltk_data]     C:\\Users\\Benoît\\AppData\\Roaming\\nltk_data...\n",
      "[nltk_data]   Package stopwords is already up-to-date!\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import nltk\n",
    "\n",
    "# Download useful tools\n",
    "nltk.download('punkt')\n",
    "nltk.download('stopwords')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Class Index\n",
    "Classe mère pour les index adaptés aux bases de données.\n",
    "Gère la lemmatization et la tokenization."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [],
   "source": [
    "from index import *"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Traitement linguistique\n",
    "Les résultats présentés pour chaque jeu de données sont reproductibles en exécutants les cellules associées.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### CACM\n",
    "#### Q1\n",
    "Nous trouvons 118 406 tokens dans la collection\n",
    "#### Q2\n",
    "Et un vocabulaire de 5 462 tokens différents\n",
    "#### Q3\n",
    "En ne lisant que la moitié des documents, on trouve un vocabulaire de 3 211 tokens différents, avec 32 463 tokens au total. On en tire les paramètres de Heap :\n",
    "\n",
    "    k : 45.14218927762959\n",
    "    b : 0.41052940261544163\n",
    "#### Q4\n",
    "En appliquant les paramètres de Heap, on peut prévoir un vocabulaire de taille 13 114 si la collection contenait 1M de tokens.\n",
    "#### Q5 \n",
    "Voir les résultats de la cellule"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "At half the docs : \n",
      "Vocabulary size : 3212\n",
      "Nb tokens : 32463\n",
      "Total\n",
      "Vocabulary size : 5463\n",
      "Nb tokens : 118406\n",
      "Heaps parameters :\n",
      "k : 45.20278539516267\n",
      "b : 0.4104302428095791\n",
      "checking the heaps parameters (the 2 following numbers should be equal):\n",
      "5463.0\n",
      "5463\n",
      "If the data had 1M tokens, it would have roughly 13114 distinct words\n",
      "('program', 2111)\n",
      "('monica', 1)\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAY4AAAEKCAYAAAAFJbKyAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDIuMi4wLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvFvnyVgAAIABJREFUeJzt3XmYXGWZ9/HvXdX7lu4knX3pJISEEBIgAYEAIiCyg4qADsgADjOOC+OO2+jIODqOMiPi6IsIIi4g24CoQGQPEiAJZCEhJGTf09nT6fRW9/vHOd1dCb1Uh6461ZXf57r6qjpPnVPnPnV1193Pcp7H3B0REZFUxaIOQERE+hYlDhER6RElDhER6RElDhER6RElDhER6RElDhER6RElDhER6RElDhER6RElDhER6ZG8qANIh4EDB3pNTU3UYYiI9Clz586tdffq7vbLycRRU1PDnDlzog5DRKRPMbPVqeynpioREekRJQ4REekRJY4ki9bv4oTv/pUXl9dGHYqISNZS4kjiDlv3NLCvsSXqUEREslbaEoeZjTSzZ8xsiZm9YWY3huX9zWymmS0LH6vCcjOzW81suZktMLPjk97rmnD/ZWZ2Tbpizs8zAJpaEuk6hYhIn5fOGkcz8AV3Pwo4CfiUmU0CbgKecvfxwFPhNsB5wPjw5wbgZxAkGuBbwHuAE4FvtSab3pYXCz4OJQ4Rkc6lLXG4+0Z3nxc+3wMsAYYDlwB3h7vdDVwaPr8E+LUHZgOVZjYU+AAw0923u/sOYCZwbjpizo8HNY7mFq2KKCLSmYz0cZhZDXAc8DIw2N03QpBcgEHhbsOBtUmHrQvLOivvdXnx4ONoTqjGISLSmbQnDjMrAx4E/sXdd3e1awdl3kX5wee5wczmmNmcrVu3HlKsrTWOJtU4REQ6ldbEYWb5BEnjt+7+UFi8OWyCInzcEpavA0YmHT4C2NBF+QHc/XZ3n+7u06uru71jvkP56uMQEelWt4kj/C/+Uz3tkDYzA34JLHH3W5JeehRoHRl1DfBIUvnHw9FVJwG7wqasJ4BzzKwqjOGcsKzX5amPQ0SkW6nMVXUlcC3wqpnNAe4CnnT37r5dZwBXAwvN7PWw7GvA94E/mNn1wBrgI+FrfwbOB5YD+8Jz4u7bzexm4NVwv++4+/ZULq6nWkdVNSeUOEREOtNt4nD35cDXzeybwIXAnUDCzO4EftzZl7i7z6Lj/gmAszrY34FPdfJed4bnTSsLo/V3dqGIiEgopT4OM5sC/Aj4L4I+i8uA3cDT6QstOt3WpUREDmPd1jjMbC6wk6C/4iZ3bwhfetnMZqQzuEyzzupHIiLSJpU+jo+4+4qOXnD3D/VyPJGysGWt++4bEZHDVypNVZ8ws8rWjXB007+nMabItPVxKG+IiHQqlcRxnrvvbN0Ip/04P30hRae1pUp5Q0Skc6kkjriZFbZumFkxUNjF/n2WWWtTVcSBiIhksVT6OH4DPGVmdxH8M34d7ZMU5pT2Gocyh4hIZ1K5j+MHZraQ4N4LA25297TcuR019XGIiHQvlRoH7v4X4C9pjiVy7U1VyhwiIp1JZa6qD4Ur7+0ys91mtsfMuprltk8zU+e4iEhXUqlx/AC4yN2XpDuYbGCoqUpEpCupjKrafLgkDQiaq9Q5LiLSuVRqHHPM7D7g/4DW6UZIWl8jp6jGISLStVQSRwXBNOfnJJU5kJuJQ30cIiJdSmU47rWZCCRbGKYah4hIF1IZVXWkmT1lZovC7Slm9o30hxYR0w2AIiJdSaVz/BfAV4EmAHdfQLAqYE6Kmfo4RES6kkriKHH3Vw4qa05HMNkgaKpS5hAR6UwqiaPWzMYR9hmb2WXAxrRGFSFTjUNEpEupjKr6FHA7MNHM1gMrgavSGlWEDI2qEhHpSiqjqlYAZ5tZKRBz9z3pDys6ZhpVJSLSlVTWHP/Xg7YBcPfvpCmmSAU1DmUOEZHOpNJUVZf0vAi4EMjdKUjUxyEi0qVUmqp+lLxtZj8EHk1bRBGz7ncRETmspTKq6mAlwNjeDiRbxGIajisi0pVU+jgW0j7QKA5UAznZvwFBjSOhvCEi0qlU+jguTHreTDDNeu7eAKhp1UVEupRK4jh4+G1F68gqAHff3qsRRUzTqouIdC2VxDEPGAnsIPherQTWhK85OdbfoWnVRUS6lkrn+OMES8cOdPcBBE1XD7n7GHfPqaQR0A2AIiJdSSVxnODuf27dcPe/AO9NX0jRClrhlDlERDqTSlNVbbj+xm8IvlGvAralNaoIqY9DRKRrqdQ4PkowBPfh8Kc6LMtJmh1XRKRrqdw5vh240czK3H1vBmKKVEzDcUVEupTK0rGnmNliYHG4PdXM/jeF4+40sy2tS86GZf3NbKaZLQsfq8JyM7NbzWy5mS0ws+OTjrkm3H+ZmV1zSFfZA7oBUESka6k0Vf038AHCfg13nw+cnsJxvwLOPajsJuApdx8PPBVuA5wHjA9/bgB+BkGiAb4FvAc4EfhWa7JJF02rLiLStZTmqnL3tQcVtaRwzPPAwTcHXgLcHT6/G7g0qfzXHpgNVJrZUIKENdPdt7v7DmAm70xGvSro41DmEBHpTCqJY62ZnQK4mRWY2Rc59GnVB7v7RoDwcVBYPhxITk7rwrLOyt/BzG4wszlmNmfr1q2HGB7kx2M0qa1KRKRTqSSOfyJYPnY4wRf3seF2b+poNnPvovydhe63u/t0d59eXV19yIEUxGM0NndboRIROWx1OarKzOLA1e7+d710vs1mNtTdN4ZNUVvC8nUE05q0GgFsCMvPOKj82V6KpUMFeTEamxPpPIWISJ/WZY3D3VsI+h96y6NA68ioa4BHkso/Ho6uOgnYFTZlPQGcY2ZVYaf4OWFZ2hTkxWhsUeIQEelMKneOv2hmtwH3kbSMrLvP6+ogM/s9QW1hoJmtIxgd9X3gD2Z2PcFEiR8Jd/8zcD6wHNgHXBueY7uZ3Qy8Gu73nXTPxhs0VSlxiIh0JpXEcUr4mLx4kwNndnWQu3d2d/lZHezrdNJv4u53And2H2bvKMiLsW9fzi43IiLyrnWaOMzsRnf/MfBNd5+VwZgiVZAXo0E1DhGRTnXVx3Ft+HhrJgLJFurjEBHpWldNVUvMbBVQbWYLksqDCWTdp6Q1sogUqo9DRKRLnSYOd/+omQ0hGMV0ceZCila+EoeISJe67Bx3903A1AzFkhXUVCUi0rWU5qo6nOgGQBGRrilxHESJQ0Ska0ocBymIx2hOOAlNdCgi0qGu7uP4I51MKAjg7jnZYV6QF+TSxpYERbF4xNGIiGSfrjrHfxg+fggYAvwm3P4osCqNMUWqMDlx5CtxiIgcrKvhuM8BmNnN7p684t8fzez5tEcWkZKC4COpa2imoig/4mhERLJPKn0c1WY2tnXDzMYAh77gRZarLAmSxc59TRFHIiKSnVKZ5PBzwLNmtiLcriFYFzwnVRYHiWPHvsaIIxERyU7dLeQUA3YD44GJYfGb7t6Q7sCiUlVaAKjGISLSme7uHE+Y2Y/c/WRgfoZiilT/MHFs25uzuVFE5F1JpY/jSTP7sJl1tP53zhlQWkDMYPNuJQ4RkY6k0sfxeaAUaDGzetpnx61Ia2QRyYvHGFRexKbd+6MORUQkK3WbONy9PBOBZJMh/YrYtEuJQ0SkI6nUODCzi4HWezmedffH0hdS9IZXFbNo/a6owxARyUrd9nGY2feBG4HF4c+NYVnOGlddxtrt+2hobok6FBGRrJNKjeN84Fh3TwCY2d3Aa8BN6QwsSuOqS0k4rN62jyMHH3YtdSIiXUp1dtzKpOf90hFINpkwJEgWr6/dGXEkIiLZJ5XE8T3gNTP7VVjbmAv8R3rDitaEweUMqSjiLws3Rh2KiEjW6TZxuPvvgZOAh8Kfk9393nQHFiUz4/xjhvLM0q26EVBE5CCpdI7fA1wIvOXuj4TrkOe8cycPAWDm4s0RRyIikl1Saaq6CxgK/MTM3jazB83sxjTHFbkTaqoYPaCEh15bH3UoIiJZJZWmqqeB7wLfBO4ApgOfTHNckTMzLp8+kldWbudXL66MOhwRkazR7XBcM3uKYMqRl4AXgBPcfUu6A8sG1586hnmrd/CdxxZTUpjH5dNHRh2SiEjkUmmqWgA0ApOBKcBkMytOa1RZoig/zn9feSynjBvIlx9YwG1PL8O902XYRUQOC6k0VX0uXDr2g8A2gj6Pw+YGh4qifO669gTeN6GaHz75FhfdNotHXl9Pc0si6tBERCJh3f0HbWafBk4DpgGrgeeBF8K+j6w0ffp0nzNnTq++Z0vC+d0ra/j5s2+zfmc9w/oVce7koVw2bQTjBpVSmBfv1fOJiGSamc119+nd7pdC4vgSQbKY6+7NvRRfWqUjcbRqSThPvrGJ372yhtkrttHU4pQX5nHGxEGcPn4gpx9ZzeCKorScW0QknXotcfRF6Uwcybbs2c/flm/j+WVbef6tWmrDmwXHDCzliEFlTB7Wj/GDyzhycDljBpYSjx0Wa2GJSB+VauJIaVr1bGBm5wI/BuLAHe4e+Qy9g8qLuPS44Vx63HBaEs7STXt4ZukWFqzbybLNe3lqyWYSYV6uLMnn6GEVjOpfwpCKYvoV5zGgrJDq8kIGlBbQv7SAypICJRcRyXp9InGYWRz4KfB+YB3wqpk96u6Lo42sXTxmTBpWwaRh7Qsj7m1oZlVtHYs37uaVldtZvmUvT7yxme11jZ2+x4DSAsqK8ijOj1NZkk9xfh6VJflUleRTWVJAcX6cfsX5lBTEKSnMo6wwTkE8TkFejLKiPArzYuTHYuTnGUV5cWJKRCLSy/pE4gBOBJa7+woAM7sXuIRgfZCsVVaYx+Th/Zg8vN8B94A0tyTYWd/E9rpGtuxuYFtdAzvqGtmyp4HavQ3UNbZQ39jCrvomdtTV88aGXezY18j+pp6N5IrHjPy4kReLkRc3CuIxSgvziMeMuBmxmJEXCx6L8mIU5seJW3Bc60/MjMK8OIX5seAYC26ODF6DWNL7FOXHg20DC1+z1m0gFkveDo8piGME+xsWPobbZuFzSypL2i+5nOBaC/LaBwqaBUnT2rbbPxsLS62DvNpaZtg7y7p4j+S3at+vg/c4YL+OYyzIi5EXS3Xy6o7O24Njen5IW9zpP88hHHMoZzqUQ7L0s279W0ynvpI4hgNrk7bXAe+JKJZ3LS8eY2BZIQPLClNe78PdaWhOUN/Yws76JvY3tVDX0MzehmaaWpzG5gR79jfR2JKgqcVpakmwd38zTeF2cyLB/qYW6psSJBLBdksCWhIJWhz2h4kqkXBaEk7Cg8cWd/Y3ttDY4rgH24mEk3Da9nGH5kSirVlORKJz4ZSh3Pax49N6jr6SODpKuQd8TZnZDcANAKNGjcpETBllFvwXUZQfp6q0IOpw3iGRcBrDe1sS3p5Y3IOk19F2U0uChuaWoAzCx9Z9gv05uJzg+LbHpGMbmltobs1ebQ/t79HKD9wlLPMDyg4cM9LBe7zjvdpf7Or9D96nsxj3NyVo6enAlUMY6HIouf5QxtMcyiCcjMV2SOfJzH9Jh3KasdWlvR/IQfpK4lgHJM/3MQLYkLyDu98O3A7BqKrMhSYQNEMVxXQvi8jhoOeNqNF4FRhvZmPMrAC4Eng04phERA5LfaLG4e7N4R3sTxAMx73T3d+IOCwRkcNSTt4AaGZbCaZHOVQDgdpeCifb5PK1ga6vL8vla4O+cX2j3b26u51yMnG8W2Y2J5W7J/uiXL420PX1Zbl8bZBb19dX+jhERCRLKHGIiEiPKHF07PaoA0ijXL420PX1Zbl8bZBD16c+DhER6RHVOEREpEeUOEREpEeUOJKY2blmttTMlpvZTVHHkyozu9PMtpjZoqSy/mY208yWhY9VYbmZ2a3hNS4ws+OTjrkm3H+ZmV0TxbUczMxGmtkzZrbEzN4wsxvD8ly5viIze8XM5ofX929h+RgzezmM9b5wxgTMrDDcXh6+XpP0Xl8Ny5ea2QeiuaJ3MrO4mb1mZo+F27l0bavMbKGZvW5mc8KynPjd7FIwUZx+CO5IfxsYCxQA84FJUceVYuynA8cDi5LKfgDcFD6/CfjP8Pn5wF8IJo48CXg5LO8PrAgfq8LnVVlwbUOB48Pn5cBbwKQcuj4DysLn+cDLYdx/AK4My38OfDJ8/s/Az8PnVwL3hc8nhb+zhcCY8Hc5HvX1hbF9Hvgd8Fi4nUvXtgoYeFBZTvxudvWjGke7tjU/3L0RaF3zI+u5+/PA9oOKLwHuDp/fDVyaVP5rD8wGKs1sKPABYKa7b3f3HcBM4Nz0R981d9/o7vPC53uAJQTT7OfK9bm77w0388MfB84EHgjLD76+1ut+ADjLggUbLgHudfcGd18JLCf4nY6UmY0ALgDuCLeNHLm2LuTE72ZXlDjadbTmx/CIYukNg919IwRfvsCgsLyz68z66w+bLo4j+K88Z64vbMp5HdhC8KXxNrDT3ZvDXZJjbbuO8PVdwACy9/r+B/gy0LoK2QBy59ogSPJPmtlcC5Z2gBz63exMn5jkMEO6XfMjR3R2nVl9/WZWBjwI/Iu777bOV0Xrc9fn7i3AsWZWCTwMHNXRbuFjn7k+M7sQ2OLuc83sjNbiDnbtc9eWZIa7bzCzQcBMM3uzi3374vV1SDWOdt2u+dHHbA6rwYSPW8Lyzq4za6/fzPIJksZv3f2hsDhnrq+Vu+8EniVo/640s9Z/7JJjbbuO8PV+BM2U2Xh9M4CLzWwVQdPvmQQ1kFy4NgDcfUP4uIUg6Z9IDv5uHkyJo12urfnxKNA6OuMa4JGk8o+HIzxOAnaF1ekngHPMrCocBXJOWBapsI37l8ASd78l6aVcub7qsKaBmRUDZxP04zwDXBbudvD1tV73ZcDTHvSwPgpcGY5MGgOMB17JzFV0zN2/6u4j3L2G4O/paXf/O3Lg2gDMrNTMylufE/xOLSJHfje7FHXvfDb9EIx6eIugjfnrUcfTg7h/D2wEmgj+e7meoG34KWBZ+Ng/3NeAn4bXuBCYnvQ+1xF0PC4Hro36usKYTiWoti8AXg9/zs+h65sCvBZe3yLgX8PysQRfjsuB+4HCsLwo3F4evj426b2+Hl73UuC8qK/toOs8g/ZRVTlxbeF1zA9/3mj9zsiV382ufvrMlCNm9jngEwRfIgsJPtz90UYlInL46RNNVWY2HPgsQYaeTHDPxZXRRiUicnjqE4kjlAcUh51mJWR555GISK7qE4nD3dcDPwTWELTl73L3J6ONSkTk8NQn7uMIRxpcQjDdwE7gfjO7yt1/k7TPDcANAKWlpdMmTpwYSawiIn3V3Llzaz2FNcf7ROIgGKK40t23ApjZQ8ApQFvicPfbCRdKmT59us+ZMyeKOEVE+iwzW53KfhltqjKzyYd46BrgJDMrCcf1n0Uw1l1ERDIs0zWOn4c31/0K+J0Hd8p2y91fNrMHgHlAM8G4915fhnFXfRNzVm0nPx6jIC9GfjxGYfiYHzcK8mIUJL1WkBcjL2Z0Mf2FiEjOyWjicPdTzWw8wc0uc8zsFeAud5+ZwrHfAr6VzvhW1tZx/d09b+JqTSityaU1qRyQZOIx8vNiFCTv01YWS3qP1mOsg7L2RNa6zzvLYpQUxCkt7CutkCLS12T828Xdl5nZN4A5wK3AcWHz09e8fR6iSBw5uIxHPz2DxuYEjS0JGpsTNLV4+JhoK29qe6217MB9mloSNLQkaGota0nQ1Ozsq2+iqYP3aAgfm1qclkTv3JB56bHD+Mp5Exnar7hX3k9EpFVGE4eZTQGuJZiffyZwkbvPM7NhwEtApImjpCCPKSMqowyBloQfkEwOTlqtyeydZe2Pq7bt457Zq3nijc188oxx3HD6WIry45Fel4jkjkzXOG4DfkFQu6hvLfRgWuJvZDiWrBSPGfFY/F1/0f/9KTV87y9LuGXmW9z36lq+ev5ELjhmqPpjRORdy+hcVeGaCvUerD+AmcWAInff15vn0XDcdrNXbOPf/riYJRt3c2JNf/71oklMHt4v6rBEJAuZ2Vx3n97dfpm+c/yvQHKje0lYJmly0tgBPPaZU/mPDx7D8q17uei2WXzlgQWs3d6ruVpEDiOZbqoq8vb1lXH3vWZWkuEYDjvxmPGx94zigilD+clTy7j7pVU8MG8dF04Zyg2nj+XoYaqBiEjqMl3jqDOz41s3zGwaUN/F/tKL+hXn840LJ/H8l9/HdTNq+OvizVxw6yyu/uXLvLi8lr4yxb6IRCvTfRwnECwh2Tqz7VDgCnef25vnUR9HanbVN/Hbl1dz56xV1O5t4Jjh/Tj/mKGM6l/CyP7FjKwqobIkXx3qIoeJVPs4Mr6QU7h+9ASC1bDedPem3j6HEkfP7G9q4eHX1vOLF1awYmvdAa+VFeYxqn8JE4aUM2FIOROHlHPU0AoGlRcqoYjkmGxOHKcANST1r7j7r3vzHEoch27P/ibWbq9n7Y59rN2+j3U76llZW8fSTXvYtLt9wcVh/Yr4wOQhnH/MUKaNqiIWUxIR6etSTRyZvgHwHmAcwbrRLWGxA72aOOTQlRflM2lYPpOGVbzjtZ37Gnlz0x7e3LibF9/exm9fXsNdL66iuryQU8YNoDg/Ht6HEvzkxYx4LEZlST4fOm44gyqKIrgiEeltme7jWAJM8jSfVDWOzNjb0MzTb27h8UUbmb92Fy0JpznhtCQS4WOw3dicoCAe45Jjh/EPp4/lyMHlUYcuIh3IyhoHsAgYQrCKn/RxZYV5XDx1GBdPHdblfqu31fHLWSv5w5y1PDBvHdfPGMMXPzBB06CI9FGZrnE8AxwLvAI0tJa7+8W9eR7VOLLTjrpGfjRzKb+ZvYZx1aXcfMlkjh7ej4qiPHW0i2SBrOwcN7P3dlTu7s/15nmUOLLbrGW1fPmB+WzYFXS2lxXmMbyymGGVRQyvKmZYZTEjqko4oaZKs/uKZFBWJg4AMxsNjHf3v4Z3jcfdfU9vnkOJI/vt2d/ErGW1rN9Zz7od9azfWc+GncHjzn3tI7SPHFzG0cP6MbJ/CedMGqx5tkTSKCsTh5n9A3AD0N/dx4WLOv3c3c/qzfMocfRtdQ3NrKyt429v1/LCslpWbK1jw6563OHMiYOYXlNF/5ICKksKGFZZxFFDK8iPZ3oSBJHck62J43XgROBldz8uLFvo7sekcGwlcAcwmWAI73Xu/lJH+ypx5J5d9U38+m+r+NXfVrGtrvGA10oK4tQMKGV4VTFfOOdIJg5551BiEeleto6qanD3xtaOUDPLI0gCqfgx8Li7XxauW67JEQ8j/Yrz+cxZ4/nMWeOpb2xhx75Gttc1srK2jjmrtrNuRz3zVu/g4tte5LoZYzhz4iAqivMYVF5ElaZNEelVma5x/ADYCXwc+Azwz8Bid/96N8dVAPOBsancA6Iax+Gpdm8D//rIIp54Y/MBS/DGY0ZpQZypIysZXFHEyWMHcPGxw9S8JXKQbG2qigHXA+cQzFX1BHBHd8nAzI4FbgcWA1OBucCN7l7X0f5KHIe37XWNvLZmB/ubEmzevZ9tdQ3s2NfE/LU72by7gdq9DQzrV8Q/nD6WK04YSUlBpiveItkpKxPHoTKz6cBsYIa7v2xmPwZ2u/s3k/a5gaDjnVGjRk1bvXp1NMFKVnN3nl26lZ89+zavrNpOPGYMqQiGAY+oDIYCj60u5aihFQzrV0y/kvyoQxbJmKxMHGa2kg76NNx9bDfHDQFmu3tNuH0acJO7X9DR/qpxSCrmrt7O029uYcPO/awPhwRv2r2/rZkrHjNOqKni7KMGM2FIeXivSbHueJecla2d48kBFQEfAfp3d5C7bzKztWY2wd2XAmcRNFuJHLJpo/szbfSBv37NLQmWbt7Dqtp9LNm4mycXb+Lf/7TkgH0GlRcydWQl500eQnV5IdNH96e4QMlEDh+RN1WZ2Sx3PzWF/Y4lGI5bAKwArnX3HR3tqxqH9KZNu/azeltd282Ka7bv45k3t7QNCy7Mi3HNKTVcccJIxlWXRRytyKHLyhpH8rKxBMvWTgdSmirV3V/nwBqLSEYM6VfEkH4HTgnf1JJg9bZ9rN9Zz/1z1nL78yu4/fkVjBlYyjHD+3HU0ApOHFPF1BGV5Gn0luSYKCY5bNUMrAJ+GDY/9RrVOCTTtuzez4Pz1jNvzQ4Wrd/FxnAerqqSfE4eN4BLjh1Ov+J8pozop1FckrWysnM8U5Q4JGpbdu9n9srtPLd0K48t2EBDcwKAgrwY1WWFVJbkM7iiiGmjqzhv8hDGqolLskBWJg4z+3xXr7v7Lb1xHiUOySbb9jawfmc9m3c3MHvFNnbUNbKrvomV2+ra1nifOKScwRVFTBhSzlFDy5kyolL9JZJxWdnHQdBHcQLwaLh9EfA8sDbDcYhkzICyQgaUFQLw/kmDD3ht/c567nt1LfPX7qR2bwMv/W0bjWHtZPygMo4YVMZRQys4+6jBTBxSrrXdJStkusbxJPDh1mnUzawcuN/dz+3N86jGIX1Vc0uClbV1PPfWVl56extvb93Lqm37ABhcUcgV00dy7KhKjhtZRVVpQcTRSq7J1hrHKCB5atNGoCbDMYhkrbx4jPGDyxk/uJxPnBbcF7thZz0vvb2NR+dv4CfPLKf1f71R/UuYNrqKY0dWMmZgKdNrqtTxLhmR6d+ye4BXzOxhgjvIPwj8OsMxiPQpwyqL+fC0EXx42ghq9zawdNMeFq7fxetrdjJreS0Pv7YegIJ4jHGDyjixpopRA0o5eewAagaWKJlIr4tiBcDjgdPCzefd/bXePoeaquRw4e5s3t3AW5v38OLyWuau3sGSjbupa2wBgmlTjhnej4umDuM9Y/ozcUi57iuRTmVrUxUE62jsdve7zKzazMa4+8oI4hDp88ys7QbF04+sbitfv7OeOau2s2zzXp59aws3P9Y+Q8+Rg8s4flQVJ9T054IpQzX3lvRYpjvHv0UwsmqCux9pZsMIOsdn9OZ5VOMQOdCKrXtZtGE3r63ZweINu1m6eQ879zVRnB9nek0VEwaXM6RfEZOGVXDSmAEavXWYytYaxweB44B5AO6+IRxZJSJpNLa6jLHVZVw8dRgQNHG99PY2nlxccs6lAAAP60lEQVS8mefe2src1TvYFzZvVZcXcvr4as6bPISjh1cwtF9xlKFLFsp04mh0dzczBzCz0gyfX0QImrhOOWIgpxwxsK2sdm8DL729jXtmr+bR+et5cN46AC44ZigzjhjI+ycNprq8MKqQJYtkuqnqi8B44P3A94DrgN+5+0968zxqqhJ5d3bvb2Lhul08vmgTD7+2nr0NzZjBJVOH8Y/vHcdRQyuiDlHSICunHAEws/eTtHSsu8/s7XMocYj0npaE8/raHTwwdz1/mLOWloQzsn8xp42v5tQjBnL6kdWUFWrIby7IusRhZnGCRHF2us+lxCGSHut31vPo6xt4ZukW5q3eQXPCiceMMycO4qMnjuTUI6opyNNw374q6xIHgJk9Clzt7rvSeR4lDpH029fYzKxltTy2YCOPzt8AQHlRHu+fNJhPvncc4wdr3Etfk62J4w/AScBMoK613N0/25vnUeIQyayd+xp5fNEm/rRwIy8sqwXg1CMGcutHj6O/5tTqM7I1cVzTUbm7353i8XFgDrDe3S/sbD8lDpHorNm2j1ufXsYDc9dRnB/n02cewd+9ZxSVJUog2S6rEoeZPeXuZ5nZf7r7V97F+3ye4AbCCiUOkez2t7dr+cqDC1i7vR6ADxw9mGtnjOE9Y/pjphsMs1G23QA41MzeC1xsZvcSjKhq4+7zunsDMxsBXAB8F+hyQSgRid4p4wby/Jfex9NvbuGe2at54o3NPPHGZgriMY4fXcmlxw7nsmkjNHdWH5SpGsdlwPXAqQRNTcnc3c9M4T0eILj3oxz4omocIn3L2u37eGzBRuau3s5fl2xpKz9t/ED+7eKjtXxuFsiqpqq2k5l9091vPoTjLgTOd/d/NrMz6CBxmNkNwA0Ao0aNmrZ69ereCFlE0mB/Uwv/99r6AzrTp4+u4ooTRvKR6SMjju7wlZWJ41CZ2feAq4FmoAioAB5y96s62l81DpG+48Xltfzn42+yYF0wSn9IRRHnHD2YT7/vCAZVFEUc3eElpxJHss5qHMmUOET6nk279vPPv53LvDU728rGDCzlqpNG8/GTR5OvvpC0U+JQ4hDpk1oSzh/nb+DeV9cwe8V2AGIGP7p8Kh88bkTE0eW2rEocZta/q9fdfXtvnk+JQyQ31DU0819PLOVXf1sFwNlHDeJ7H5qiWXrTJNsSx0qCNcYNGAXsCJ9XAmvcfUxvnk+JQyS3rNi6l4t+MqttSdxTjxjILVdMZVC5+kB6U6qJIyONhu4+xt3HAk8AF7n7QHcfAFwIPJSJGESk7xpbXcaCb3+AWy6fSkFejFnLaznxu0/xjf9bSF1Dc9ThHXYyPRx3rrtPO6hsTioZridU4xDJXe7OL2et5N//tKSt7OqTRvPlcydQXpQfYWR9X1bVOJLUmtk3zKzGzEab2deBbRmOQUT6MDPjE6eN5c2bz+UzZx4BwD2zV3PMt5/kE3e/yr5G1UDSLdM1jv7At4DTCfo8nge+o85xETlU7s7vX1nL1x5e2Fb2T+8dx1fOnaA5sXooqzrHoW1m2++7+5fSfS4lDpHDTyLh/M9f3+LWp5e3lX3qfeP4/PsnEI8pgaQi6xIHgJk9ncq8VO+WEofI4WtfYzP/PfMtfvHCSgDKCvM4fnQV3710MiOqilUL6UK2Jo4fAeOB+zlwIadeHVmlxCEiO+oa+drDC3nura3sC4fxnjlxED/92PEUF8Qjji47ZWviuKuDYnf363rzPEocItJqf1MLTy7ezGd//xoQ3IV+/z+dzPGjqlT7OEhWJo5MUeIQkYO1JJwv/OF1/u/1YH30SUMr+MFlU5g8vF/EkWWPrByOa2YjzOxhM9tiZpvN7MFwgSYRkbSKx4z/ufI4br96GkX5MRZv3M2FP5nF9//yJht21kcdXp+S6fs47gIeBYYBw4E/hmUiIhlxztFDmPON9/NvFx8NwM+fe5sZ//k0r67q1bsCclqmE0e1u9/l7s3hz6+A6gzHICKHubLCPK45pYY3bz6XC6YMxR0+8vOX+NrDC1kYrgsinYvizvGrzCwe/lyF7hwXkYgU5cf56ceO5yvnTmRgWSH3vbqWL94/nx88/ib14UgseadMJ47rgMuBTcBG4LKwTEQkMp88YxxzvnE2V5wwkvU76/nfZ9/m3/+0mMcXbYo6tKyU6eG4Re6+P93n0agqETlUO+oaOe0Hz7C3oRkz+N0nTmLUgBKGVxZHHVraZeWoKmCRmb1oZt83s/PNTOPgRCSrVJUWMPebZ3PL5VNxh4/+Yjbn3PIcTS2JqEPLGhlNHO5+BPBRYCHBWhzzzez17o4zs5Fm9oyZLTGzN8zsxnTHKiKHr8K8OBdPHcbvPvEerp1RQ11jC2f96Dk+d1+3X1eHhbxMniy8Z2MGcBowFXgDmJXCoc3AF9x9npmVA3PNbKa7L05ftCJyOMuLxzjliIGMrS5jz/5mFqzbyWMLNnD2UYMpzItx+pHVFORlutEmO2S6jyMBvAr8h7s/8i7e5xHgNnef2dHr6uMQkd527ytruOmh9qnbf37V8Zw7eWiEEfW+rJxyxMymAqcSrMcxClgGPOfuv+zBe9QQrOMx2d13J5XfANwAMGrUqGmrV6/uvcBF5LDn7qyorWPb3kYu/38vMX10FeOqyzh6eAUfP7km6vB6RVYmDgAzKyNIHqcBVxFMcljTg2OfA77b1Yy6qnGISLq0JJwrb3+Jtdvr2bO/iRZ33rz5vKjD6hWpJo5M93HMAQqBvxH0bZzu7ilVDcwsH3gQ+G1vT8MuIpKqeMy4/59OAeC2p5fxwyff4gt/mI8ZGHD1yaOZMqIy2iDTLKOJAzjP3bf29CAL5j7+JbDE3W/p/bBERHpuek1/Rg8oYfaKYAKMjbvqKcyPKXH0pkNJGqEZwNXAwqThu19z9z/3TmQiIj130tgBPPel97Vtn/aDp1lVu48XlrV/1Y3qX8LoAaVRhJc2ma5xHBJ3n0VQCxQRyVrVZYXMWl7LrOW1bWVD+xXx0lfPijCq3tcnEoeISF/wi49PZ2Vt26rY3DN7dU7Od5XpzvEPdVC8C1jo7lsyGYuISG8bUFbIgLLCtu0XltXS0Jzg3lfWkLxKbVF+nPOPGUp+vG/eQJjpGsf1wMnAM+H2GcBs4Egz+46735PheERE0mZk/xKAA24cbNWvOJ8zJgzKdEi9ItOJIwEc5e6bAcxsMPAz4D0EN/UpcYhIzrhs2ghOHz+Q5kT7/XIra+v4uztepq6h7673kenEUdOaNEJbgCPdfbuZNWU4FhGRtBtUUXTAdussu/ublDhS9YKZPQbcH25fBjxvZqXAzgzHIiKScYV5cQDufHElMxdv7nCfvLjxxXMmUDMwO4fxZjpxfAr4EMGUIwbcDTzowbwn7+vqQBGRXDCgrID3HlnNpl37DxiB1ao5keDtrXVMH13F3w8cE0GE3cv0DYBuZrOARsCBVzzTk2WJiEQoPx7j7utO7PT13fubmPLtJw/oF8k2GR0LZmaXA68QNFFdDrxsZpdlMgYRkWxWEA7RbWrJ3sSR6aaqrwMntN6zYWbVwF+BBzIch4hIVspvSxzZu1RtphNH7KAb/baR+XXPRUSyVjxmxAxeXbWdO15Y0ePjx1aXcubEwWmIrF2mE8fjZvYE8Ptw+wpAExWKiCSpGVDKC8tqeWFZbfc7H+TCKUNzK3G4+5fM7MMEs90acLu7P5zJGEREst0Tnzud+kO8zyM/lv5GnIxPcujuDxIsyCQiIh3Ij8eyeh6rjCQOM9tDMPz2HS8RjNKtyEQcIiLy7mUkcbh7eSbOIyIi6We5eP+dmW0FUlrLvBMDgZ73SuU+fS6d02fTOX02HcvGz2W0u1d3t1NOJo53y8zmuPv0qOPINvpcOqfPpnP6bDrWlz+X7O19ERGRrKTEISIiPaLE0bHbow4gS+lz6Zw+m87ps+lYn/1c1MchIiI9ohqHiIj0iBJHEjM718yWmtlyM7sp6niyhZmNNLNnzGyJmb1hZjdGHVM2MbO4mb0Wrm4pITOrNLMHzOzN8Hfn5KhjyhZm9rnwb2mRmf3ezIq6Pyp7KHGEzCwO/BQ4D5gEfNTMJkUbVdZoBr7g7kcBJwGf0mdzgBuBJVEHkYV+DDzu7hOBqegzAsDMhgOfBaa7+2QgDlwZbVQ9o8TR7kRgubuvcPdG4F7gkohjygruvtHd54XP9xB8AQyPNqrsYGYjgAuAO6KOJZuYWQVwOvBLAHdvdPed0UaVVfKAYjPLA0qADRHH0yNKHO2GA2uTttehL8d3MLMa4Djg5WgjyRr/A3wZyN5Vd6IxFtgK3BU2491hZqVRB5UN3H098ENgDbAR2OXuT0YbVc8ocbSzDso05CyJmZURzGz8L+6+O+p4omZmFwJb3H1u1LFkoTzgeOBn7n4cUAeo3xAwsyqC1owxwDCg1MyuijaqnlHiaLcOGJm0PYI+Vn1MJzPLJ0gav3X3h6KOJ0vMAC42s1UETZtnmtlvog0pa6wD1rl7a830AYJEInA2sNLdt7p7E/AQcErEMfWIEke7V4HxZjbGzAoIOqsejTimrGBmRtBWvcTdb4k6nmzh7l919xHuXkPw+/K0u/ep/xzTxd03AWvNbEJYdBawOMKQsska4CQzKwn/ts6ijw0cyPhCTtnK3ZvN7NPAEwSjHO509zciDitbzACuBhaa2eth2dfcXcv+Slc+A/w2/EdsBXBtxPFkBXd/2cweAOYRjFh8jT52F7nuHBcRkR5RU5WIiPSIEoeIiPSIEoeIiPSIEoeIiPSIEoeIiPSIEodIN8xsb9QxQDDdi5ktijoOESUOkQiEk9uJ9ElKHCIpssB/hWsoLDSzK8LymJn9b7i+wmNm9mczu6yD4581s/8ws+eAG83sIjN7OZwE8K9mNjjc79tmdme4/woz+2wH7zU2PO6EtF+4yEH0X49I6j4EHEuwtsRA4FUze57gzvoa4BhgEMH0EXd28h6V7v5eaJvs7iR3dzP7BMEsu18I95sIvA8oB5aa2c9a3yCcxuNe4Fp3fx2RDFPiEEndqcDv3b0F2BzWHE4Iy+939wSwycye6eI97kt6PgK4z8yGAgXAyqTX/uTuDUCDmW0BBofl1cAjwIc1JY5ERU1VIqnraOr9rso7Upf0/CfAbe5+DPCPQPLyoQ1Jz1to/ydvF8G6MTN6cE6RXqXEIZK654ErwjXGqwlWuHsFmAV8OOzrGAyckeL79QPWh8+vSfGYRuBS4ONm9rGUIxfpRWqqEkndw8DJwHyCRb6+7O6bzOxBgqmxFwFvEayOuCuF9/s2cL+ZrQdmEyzs0y13rwsXkZppZnXu/kiPr0TkXdDsuCK9wMzK3H2vmQ0gqIXMCNekEMk5qnGI9I7HzKySoJP7ZiUNyWWqcYiISI+oc1xERHpEiUNERHpEiUNERHpEiUNERHpEiUNERHpEiUNERHrk/wMBybwa70UQIwAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from q1_to_5_CACM import analysis\n",
    "analysis(path_to_cacm_file)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### CS276\n",
    "Les résultats présentés ici sont reproductibles en exécutants les cellules suivantes.\n",
    "\n",
    "#### Q1\n",
    "Nous trouvons 17 012 440 tokens dans la collection\n",
    "#### Q2\n",
    "Et un vocabulaire de 206 430 tokens différents\n",
    "#### Q3\n",
    "En ne lisant que la moitié des documents, on trouve un vocabulaire de 3 211 tokens différents, avec 32 463 tokens au total. On en tire les paramètres de Heap :\n",
    "\n",
    "    k : 0.05036578468663431\n",
    "    b : 0.9145139883303451\n",
    "    \n",
    "#### Q4\n",
    "En appliquant les paramètres de Heap, on peut prévoir un vocabulaire de taille 15 460 si la collection contenait 1M de tokens.\n",
    "#### Q5 \n",
    "Voir les résultats de la cellule\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Checking the saved results without reading again the dataset\n",
      "Question 1 : 17012440 terms in the vocabulary\n",
      "Question 2 : 206430 distinct tokens\n",
      "Question 3 : Heaps Law : k = 0.05036578468663431 and b = 0.9145139883303451\n",
      "Question 4 : For 1 million tokens, the vocabulary size would be 15460\n",
      "Question 5\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAZsAAAEaCAYAAADHdPqFAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3XmYFNW5x/Hvjx0UWQQV2RHEqHFBFKO5LnFDTdQYTSQmLsEt6jUmJjcuSSSLuWYxLokacYm7BJUoRgnifuOGIIgLEiagAiL7JiLre/84pzNF2z3dw0x39cy8n+epZ6pPbW9VTffbder0KZkZzjnnXCk1SzsA55xzjZ8nG+eccyXnycY551zJebJxzjlXcp5snHPOlZwnG+eccyXnycY1epJGSLq3ntbVVtJjklZIerCI+U+WNEFSm/rYvnMNlScbV3aSTpH0qqTVkhbG8fMlKe3YinASsD2wrZmdXNOMkvYGzgJOMLNPyxGcc5XKk40rK0mXANcDvwN2IHxwnwccCLTKs0zzsgVYWG/gX2a2IddESS0y42Y2xcyOMrPVZYuugaqwc+xKwJONKxtJHYBfAOeb2UNmtsqCKWZ2qpmtjfPdKelmSU9IWg0cKulYSVMkrZQ0R9KIxHr7SDJJ50j6UNJ8ST/M2nwrSXdLWiXpbUmDE8vvKOlhSYskzZZ0UZ74fw78DPiGpI8lDZd0hqQXJV0raQkwIs77HUnTJS2TNF5S78R6jpD0bqyK+5Ok5yWdFadtVuWX2LcWmWMo6fa4j/Mk/SrzQR1j+aek38ftzpZ0dGJdnSX9JR6jZZIeSUz7sqSpkpZLeknSHnmOwc2Sfp9V9qikH8TxH8e4VkmaIemwPOvZ0nN8uqQPJC2WdEVieltJd8X9mi7pfyTNzbVtlxIz88GHsgzAUGAD0KLAfHcCKwhXO82ANsAhwOfj6z2ABYTqKYA+gAEPAFvF+RYBh8fpI4BPgWOA5sD/Aq/Eac2AyYQk0groB8wCjsoT2wjg3sTrM+I+/TfQAmgLHA9UAZ+LZT8BXorzdwFWEarjWgLfj8uflWf9mX1rEV//Dbgl7ud2wETg3EQs64Gz435+F/gQUJz+OPBXoFPc9sGxfG9gITAkLnc68B7QOsf+HwTMSayzE7AG2BEYGKftmIh9p3o+x7fGY7wnsBb4XJx+NfB8jKcHMA2Ym/b/vA+Jc552AD40nQH4FvBRVtlLwPL4gXVQLLsTuLvAuq4Dro3jmQ+iXRLTfwvcHsdHAE8lpu0KrInjQ4APstZ9GfCXPNvNTgZn5Fh+HDA88boZ8AmhCu40YqKL0wTMpYhkQ6hyXAu0TUwfBjybiKUqMa1dXHYHoBuwCeiUY59uBn6ZVTaDmIyyygV8kDhXZwPPxPH+hKR1ONCywPnb0nPcIzF9InBKHN/sCwLhXpknmwoavBrNldMSoEvWfY0DzKxjnJb8f5yTXFDSEEnPxqquFYT7PF2y1p9c5n3Ct+2MjxLjnwBtYhy9gR1j9dFyScuBywkf7MWak/W6N3B9Yn1LCR/S3WNM/5nfwidj9vL59CZckcxPrPsWwhVOxn/208w+iaNbAz2BpWa2LM96L8k6Bj3Z/Pgl4x1FSHIA3wTui9OqgIsJCXOhpFGSPrOOhC05x9nnces4vtlxzV63S58nG1dOLxO+mR9fxLzZ3ZHfD4wFeppZB+DPhA/wpJ6J8V6EKqRC5gCzzaxjYmhvZscUsWy+WOcQqraS62xrZi8B85NxSlJW3KsJVyQZO2Stdy3QJbHebcxstyL3s7OkjnmmXZUVbzszeyDPuh4ATor3oYYAD2cmmNn9ZvZFQgIz4Dc1xLQl5zif+YTqs4ye+WZ06fBk48rGzJYDPwduknSSpPaSmknai3APoibtCd/MP5W0H+EbdbafSmonaTfgTML9iUImAqvije22kppL2l3SvrXYtWx/Bi6LcWRu6meaST8O7CbpxHhldRGbJ5SpwEGSeik0qLgsM8HM5gNPAtdI2iYeu50kHVwooLjsOMKx7ySppaSD4uRbgfPilYUkbRVv1rfPs64pwGLgNmB8PK9IGijpS5JaE+6RrSFU3RWrmHOcz2jCMe8kqTtwYS2WdWXgycaVlZn9FvgB8D+EG8ALCFVBPybcv8nnfOAXklYRbuaPzjHP84Qb808DvzezJ4uIZyPwZWAvYDbVH6IditylXOv8G+Eb/ShJK4G3gKPjtMXAyYQb2kuAAcCLiWUnEJLkNELDhb9nrf40QkOGd4BlwEOE+zHF+DahAcG7hHsrF8dtTiLce/lTXGcV4f5PTe4n3Ju5P1HWOu7XYkJ113YkkmURijnH+fyCcO9rNvAU4bisrcXyrsQyLUqca7Ak9SF8yLS0PL9/qWSSniM0Crgt7VgaC0nfJTQeKHjV58rDr2yccw2epG6SDoxViwOBSwjNxF2FaFF4Fuecq3itCNWxfQlN6UcBN6UakduMV6M555wrOa9Gc845V3KebJxzzpWc37OJunTpYn369Ek7DOeca1AmT5682My6FprPk03Up08fJk2alHYYzjnXoEh6v5j5vBrNOedcyXmyqaPrr4f27eEnP0k7Euecq1yebOpo3Tr4+GNY6x1jOOdcXp5s6on/XMk55/LzZFNHKrYDdOeca8I82TjnnCs5Tzb1xKvRnHMuP082deTVaM45V5gnG+eccyXnyaaeeDWac87l58mmjrwazTnnCvNk45xzruQ82dQTr0Zzzrn8PNnUkVejOedcYSVNNpLaSJoo6Q1Jb0v6eSzvK+lVSVWS/iqpVSxvHV9Xxel9Euu6LJbPkHRUonxoLKuSdGmiPOc2nHPOlV+pr2zWAl8ysz2BvYChkvYHfgNca2b9gWXA8Dj/cGBZLL82zoekXYFTgN2AocBNkppLag7cCBwN7AoMi/NSwzZKwqvRnHMuv5ImGws+ji9bxsGALwEPxfK7gBPi+PHxNXH6YZIUy0eZ2Vozmw1UAfvFocrMZpnZOmAUcHxcJt826pVXoznnXGElv2cTr0CmAguBCcC/geVmtiHOMhfoHse7A3MA4vQVwLbJ8qxl8pVvW8M2nHPOlVnJk42ZbTSzvYAehCuRXUq9zWJJOkfSJEmTFi1aVKd1eTWac87lV7bWaGa2HHgW+ALQUVKLOKkHMC+OzwN6AsTpHYAlyfKsZfKVL6lhG8mYRprZYDMb3LVr1y3aL69Gc865wkrdGq2rpI5xvC1wBDCdkHROirOdDjwax8fG18Tpz5iZxfJTYmu1vsAAYCLwGjAgtjxrRWhEMDYuk28bzjnnyqxF4VnqpBtwV2w11gwYbWZ/l/QOMErSr4ApwO1x/tuBeyRVAUsJyQMze1vSaOAdYANwgZltBJB0ITAeaA7cYWZvx3X9OM82SsKr0ZxzLr+SJhszmwbsnaN8FuH+TXb5p8DJedZ1FXBVjvIngCeK3UZ982o055wrrFbVaJJ6Szo8jreV1L40YTnnnGtMik42ks4m/G7llljUA3ikFEE1RF6N5pxz+dXmyuYC4EBgJYCZzQS2K0VQDYlXoznnXGG1STZr46/0gf80Tfbv88455wqqTbJ5XtLlQFtJRwAPAo+VJqyGx6vRnHMuv9okm0uBRcCbwLmEFmA/KUVQDYlXoznnXGG1afrclvA7llsh9HkWyz4pRWDOOecaj9pc2TxNSC4ZbYGn6jechsur0ZxzLr/aJJs2iccFEMfb1X9IDYtXoznnXGG1STarJQ3KvJC0D7Cm/kNqmPzKxjnn8qvNPZuLgQclfQgI2AH4RkmiakD8ysY55worOtmY2WuSdgEGxqIZZra+NGE555xrTGrbEee+QJ+43CBJmNnd9R5VA+TVaM45l1/RyUbSPcBOwFRgYyw2oEknG69Gc865wmpzZTMY2DU+mMw555wrWm1ao71FaBTgcvAU7Jxz+dUm2XQB3pE0XtLYzJBvZkk9JT0r6R1Jb0v6XizvLGmCpJnxb6dYLkk3SKqSNC2rmfXpcf6Zkk5PlO8j6c24zA1SqNTKt41S8Go055wrrDbVaCNque4NwCVm9np8yNpkSROAM4CnzexqSZcS+lz7MXA0MCAOQ4CbgSGSOgNXEqrxLK5nrJkti/OcDbxK6KttKDAurjPXNpxzzqWg6CsbM3seeA9oGcdfA16vYf75ZvZ6HF8FTAe6A8cDd8XZ7gJOiOPHA3db8ArQUVI34ChggpktjQlmAjA0TtvGzF6J95HuzlpXrm2UjFejOedcfnV5Umd3inxSp6Q+wN6EK5DtzWx+nPQRsH1ifXMSi82NZTWVz81RTg3byI7rHEmTJE1atGhRMbuSYx1btJhzzjUpJX9Sp6StgYeBi81sZXJavCIp6TVBTdsws5FmNtjMBnft2rWUYTjnXJNW0id1SmpJSDT3mdmYWLwgVoER/y6M5fOAnonFe8Symsp75CivaRsl49VozjmXX8me1Blbht0OTDezPyQmjQUyLcpOBx5NlJ8WW6XtD6yIVWHjgSMldYqtyo4ExsdpKyXtH7d1Wta6cm2j3nk1mnPOFVab1miXAsPZ/Emdt9Uw/4HAt4E3JU2NZZcDVwOjJQ0H3ge+Hqc9ARwDVBEeyHYmgJktlfRLQoMEgF+Y2dI4fj5wJ+HZOuPiQA3bcM45l4LadMS5Cbg1DsXM/09C79C5HJZjfiPcF8q1rjuAO3KUTwJ2z1G+JNc2Ssmr0ZxzLr/a9I02mxz3aMysX71G1MB4NZpzzhVW277RMtoAJwOd6zcc55xzjVFtftS5JDHMM7PrgGNLGFuD4tVozjmXX22q0QYlXjYjXOnU9nk4jY5XoznnXGG1SRbXJMY3ELqu8VZezjnnCqpNa7RDSxlIQ+fVaM45l19tqtF+UNP0rB9uNhlejeacc4XVtjXavoRf5wN8BZgIzKzvoJxzzjUutUk2PYBB8XEBSBoBPG5m3ypFYA2NV6M551x+tekbbXtgXeL1OvJ03d+UeDWac84VVpsrm7uBiZL+Fl+fQPUDypxzzrm8atMa7SpJ44D/ikVnmtmU0oTV8Hg1mnPO5VebajSAdsBKM7semCupbwlialC8Gs055wqrMdlI2j0xfiXwY+CyWNQSuLd0oTUsfmXjnHP5Fbqy6SXp6jj+VeA4YDWAmX0ItC9hbM455xqJGu/ZmNkTkjbGl+vMzCQZgKStSh5dA+DVaM45V1jBezZmNj6OjpZ0C9BR0tnAUxR4kJqkOyQtlPRWoqyzpAmSZsa/nWK5JN0gqUrStGTHn5JOj/PPlHR6onwfSW/GZW6Ij4fOu41S8mo055zLrzaPGPg98BDwMDAQ+JmZ/bHAYncCQ7PKLgWeNrMBwNPxNcDRwIA4nAPcDCFxAFcCQ4D9gCsTyeNm4OzEckMLbKPe+ZWNc84VVlSykdRc0rNmNsHMfmRmPzSzCYWWM7MXgKVZxcdT/fucuwi/18mU323BK4QrqG7AUcAEM1tqZsuACcDQOG0bM3slPlL67qx15dqGc865FBSVbMxsI7BJUod62Ob2ZjY/jn9EdS8E3YE5ifnmxrKayufmKK9pG5uRdI6kSZImLVq0aAt3J/BqNOecy682PQh8DLwpaQKxRRqAmV20pRtPNjgolZq2YWYjgZEAgwcP3qI4vBrNOecKq02yGROHulogqZuZzY9VYQtj+TygZ2K+HrFsHnBIVvlzsbxHjvlr2oZzzrkUFKxGk9QLwMzuyjVswTbHApkWZacDjybKT4ut0vYHVsSqsPHAkZI6xYYBRwLj47SVkvaPrdBOy1pXrm2UjFejOedcfsXcs3kkMyLp4dqsXNIDwMvAQElzJQ0HrgaOkDQTODy+BngCmAVUEZpUnw9gZkuBXwKvxeEXsYw4z21xmX8D42J5vm3UO69Gc865woqpRkt+nParzcrNbFieSYflmNeAC/Ks5w7gjhzlk4Ddc5QvybUN55xz6SjmysbyjLsEr0Zzzrn8irmy2VPSSsIVTts4TnxtZrZNyaJrALwazTnnCiuYbMyseTkCcc4513jV9nk2Lg+vRnPOufw82dSRV6M551xhnmycc86VnCebeuLVaM45l58nmzryajTnnCvMk41zzrmS82RTT7wazTnn8vNkU0dejeacc4V5snHOOVdynmzqqEXsg2H9+nTjcM65SubJpo7atw9/V61KNw7nnKtknmzqKJNsVq6seT7nnGvKPNnU0Y47hr9z5qQbh3POVbJGnWwkDZU0Q1KVpEtLsY2ePaFdO5g3Dz78sBRbcM65hq+Y59k0SJKaAzcCRwBzgdckjTWzd+pzOy1awOGHw9ixcNxxcMYZ0Ls3dOgALVt+dmjWrDKaS6cdQ9rb9xgqY/seQ2XE0KwZ7LBDabfRaJMNsB9QZWazACSNAo4H6jXZAFx7Lbz6KkyeHAbnnGtItt8ePvqotNtozMmmO5C8kzIXGJKcQdI5wDkAvXr12uIN9esH774Lo0eHpDN/fmidtn79Z4dNm7Z4M/Um7d4O0t6+x1AZ2/cYKieGrl1Lv43GnGwKMrORwEiAwYMH1+l0d+wI55wTBuecc5trzA0E5gE9E697xDLnnHNl1piTzWvAAEl9JbUCTgHGphyTc841SbK0KwtLSNIxwHVAc+AOM7uqhnkXAe/XYXNdgMV1WL4p8GNUHD9OxfHjVFg5jlFvMyt416dRJ5tykjTJzAanHUcl82NUHD9OxfHjVFglHaPGXI3mnHOuQniycc45V3KebOrPyLQDaAD8GBXHj1Nx/DgVVjHHyJNNPYm/2WmyJI2QdG9N8xR7jCS1lfSYpBWSHixi/pMlTZDUpth4K1lj+l+S9ICkE+q4jpz/W8Ucp+SykvpIMkm1/n1hMf/fRa5nD0kv1XU9xaqk/yVPNo2UpFMkvSpptaSFcfx8qRJ6gSroJGB7YFszO7mmGSXtDZwFnGBmn5YjOFccSXsAewKPph1LpTCzacBySV9JO5Zy82TTCEm6BLge+B2wA+GD+zzgQKBVnmWaly3AwnoD/zKzDbkmJr+ZmtkUMzvKzFaXLboGKoVzfC5wn3mT12z3EY5N02JmPtRhAIYCM4Aq4NIKiKcDsBr4WoH57gRuBp6I8x8OHAtMAVYS+pUbkZi/D2CEvuQ2AOsJPTJMitN/A3wUl90ATAcGAwJuAGYDK4Blcfwi4HRgZhxOj+v5eVy3AZuAp4AzgBeBa4ElwK/ivN+J21kGjCe098/EewTwbtzmn4DngbPitBHAvTn2rUXiGN4OzI/7+CugeZx2BvBP4PeJfTk6TrsDWBTLP4x/HwcmxH2cArwJLAdeInzoVAHTgEGJeJ6OyyaPy6PAH+Lyi4GPgVXxf++ExDYmAJ3qeI5PBz6I27kiMb0tcFeMbTrwP8DcGv7HZgFfzCr7Ttz/TTH+3olzsiCWbYjbuDxOeyqWrwA+Ad4m/G9l3nuz43FZFMdHAK/GY/sWcH9cz85x/6ri9M/nO8859mUEm//PHBfjWA48B3wuMW1QPMargAeBvxL/Z+P07sAaoHUNx64n8Cyh4+C3ge/F8s55znXmfZbr/+kz77NYvk88blVxWdW0jTp/NqX94diQB8KPRf8N9CNcMbwB7JpyTEPjm7VFgfnujG/eAwlXuG2AQ+IbsBmwR3zznxDn7xPfqA8Qfvz6X/HNfXic/iIhSRwDXAa8DLwSX48DJhNuVk6Mx+u9+CbvDHQifDBl3jjzgH/EN9A44Jq4T/9N6M+vLaEH7yrgc7HsJ8BLcfku8Y1+EtAS+H5cvthk8zfgFmArYLsY87lx2hlxP8+O5/+7hMQi4CDg/+Jx7RS3/QBwKbA3IUHcHZe7hvDB2RrYH3g1rr9zXN+8xHHpQ/hwmgJ8nZAkngGOjtNGEr/oxG39po7n+NZ4jPcE1hI/SIGrCUm7E6H7p2nkSTbx2BnQNVGWOWffBvaN286cs18TEuAlMc72hI5zdyV8ifkUOJPwfrua8L/1b2An4PU4z56E/62PgV/H9U5KHNufZM4zoUeRufnOc479GUH8nyEkrdWELzQtCUm3ivAZ0Irw/vhenHYisI5EsonrWAnsUcP7sxsxYcRj8a94LH6b51xn3mfis/9Ps8j9PpsY5828zzJfmnJuo86fTWl+MDb0AfgCMD7x+jLgspRj+hbwUVbZS4RvYGuAg2LZncDdBdZ1HXBtHO8T36i7EBJFl/hPeXucvhh4IY53I3zDXBPfzD8BPojTZsTpDwDvJrZ1CzAsTluceGMPI3zD+yArtnHA8MTrZoQP797AacAriWkifLAUTDaEKse1QNvE9GHAs3H8DMKjKzLT2sVld4ixbwLeSUzP7O/NhCuzGYn9/RA4OGu+YXHaB4TkdQshmfyTcKXWH1gIXAXcmlw2cewz29jSc9wjMX0icEocnwUclZh2FvmTTfe4rja5zlnc1luJc/YQMC/Hei4jXNk8FV+Pj8dobRwfEo/VZXEQIRHcGee/Dfgwjj+fOM87xvGc5zlHHP/5nwF+CozO+t+bR0jkB8VxJab/k88mm3nE92KR7+tHCckt37m+BRiW4/9uGHBLnvfZu1n7fktN/091HZp0r8/1oOBjDFKwBOgiqYXFex5mdgCApLlsfp9us4dZSxpC+Na4O+EbWmtCNQBZyxjwJCHhrIzlWxPe9BC+ZW5L+Ibag3BcdpS0nPAt8l+Eb33J7oHmEo5n98Q6M+WdsmMlfEBdL+ma5C7E5XdMzm9mJqnYB3f3jrHNT7SlaJa1/f88+cPMPonzbU349pipIsrY3szmS+pN+DBqmzgOG2Osmf3M7P8cYBThA2AuoRrkfuAAM6uSdDHhG+cuktoDO5jZ/ERs2ye2vyXnOPlkk0/ivkHWcc1ed5bl8W97wlUJbH7OmsVjsC7ucwego6RphKuRS8xsGdX/D5/EdcwFOsbY58Z17kj4QtOCcFzaEq5UiMu2i+PJx4P1yOyrJIvj2ec5nx1J/O+a2ab4/9WdcE7nWfykjnKtsz3Vx6hGkvoQroxfJf4/ZWKn+lzn+izqXqB8bo5yathGnXgDgcbnZcK3vuOLmNeyXt9P6Ky0p5l1AP5M+ABP6kmohx9E+LbVTdJBm600vNGS614IzDazjsALwKGED4d7itqj3LHOIVR5dEwMbc3sJUL13H96/I4t8JI9gK+m+gMINv8QmkM4fl0S693GzHYrIsY5hA/CXO+rOYSrkeXxOIwnVEE+kGddDxCqATsAvQjVZgCY2f3ABbHMCB+umWnZx35LznE+86n+kIbNj+lmLDTY+DehyinjP+eMUIU3PXHOHiZ8o94rbucaijOHcBV9HqH6bydglpkdU8RyBuy8Bef5Q0KSAzb7/5oXY++e1epzs+MkqTshWc4otCFJWxOOzcVmlvwSlutc17v63IYnm7qpuMcYmNlywk32mySdJKm9pGaS9iJ8k6xJe2CpmX0qaT/gmznm+SmwTNJuwDcIN5/3I9STtwGQlKkKg3A8VgGrJP2YcLzmEz7gBiXWmzl284BtssqX5Yjjz8BlMQ4kdZCUaSb9OLCbpBNjy7WL2DyhTAUOktRLUgdC9QsA8Rvdk8A1kraJx24nSQfnPmTV4rLPEa7iOklqCayMx+NW4Pz4WoT7FV+JVybZ+9/TzKbEY3gK4V7gDKCHpIGSvkSohppDqKpcE7eROfYLawizmHOcz2jCMe8UPzAvLDD/E0DyuG12zoBmiXN2H6HK5iJCI4T941VYrv+HzJVXT0I136q4HwsI/yvbSto/zrMN1VdFySu2RYR7b5fX9jwTjsOxkg6L5/gSwheUlwhf9jYCF0pqIel4wvsj6WDgGTNbW9NG4rofJrToGxOLF+Q51/k+i2oq75GjvKZt1Iknm7qpyMcYmNlvgR8QblwuiMMtwI8Jb4h8zgd+IWkV8DPCmyrby4RvrE8TmlfvRKh7nxHHIVT7TIjjjxFuCH8ZOIzquvqvA1+IH1ydgCMJ97/mE68s4ofyaYQb49n7+DdCC7hRklbGdR4dpy0GTiZUFy0BBhAaMGSWnUBoITSN0HDh71mrP43wzfMdwofXQ4QPwmJ8n/BN8F3Cm/RTQgugSYSrmVZxnV+N2yF+MK6I+z4eODIekzGEqo2b4rSVhFZYVwN/Ify/bUf4oD49bv90av5dSzHnOJ9fEKpbZhPuozxEOFf5jAROzXzLT54zQiuoAcRzRqiqOwL4CuHLQF/CFfBYQpVfM0l94zJvxGX6E676vkq40f09QoLeRPh/g3CllKkueioR20mEZFjr82xmMwj3Rv8Yt/cV4Ctmts7M1hEaBQwnVJN9i/D/lTxOpxISb17xmN1OuPr7Q2LSWHKf67HAaQpy/j/leJ+tlLR/4n32aIFt1E193PhpygOhFci/CB/AV6QdTwn3sw/hQzTzZn+D0CTzijh9W0ICmkl4U3eO5QJujMfnTWBwYp3fIbTiqQLOTJQPJiSPfxOaLase4n+O2ECghMfoAcJV23rCB9zwchyXfNsow//Ed4HnC8xzP7G1W4HjdE88DtMIH3bdEvNfEfd5BrHFVCzP+d4jtEibGI/fg8QmxoQr7wdj+USgX5mO06uZ80ioPny5iGW+GN9v0wjJd2rc3wb7/+SPGHBFiTcpZwMtLc+PLSuZpOcIrYluSzuWhipWqfQjXN0OIFRX/snMrks1sAoTq+JmEK56Mlcx/az6pnuT5K3RnHPFakWoju1LqCIaBdyUakSVaSChenIrQnPxk5p6ogF/eJpzzrky8AYCzjnnSs6r0aIuXbpYnz590g7DOecalMmTJy82s66F5vNkE/Xp04dJkyalHYZzzjUokt4vPJdXoznnnCsDTzZ1NH8+XHstLFmSdiTOOVe5PNnU0Z13wg9+ADvuCMOGwTPPwKZNBRdzzrkmxZNNHQ0eDEOHwvr1MGoUHHYY7LwzXH01fPRR4eWdc64p8GRTR0ccAePGwXvvwZVXQo8e8O9/w2WXhfGvfhWeeAI2bkw7UuecS48nm3rSqxeMGBGSzuOPwwknhPJHHoFjj4W+fcP0Dz6oYSXOOddIebKpZ82bwzHHwN/+BnPmwK9/Df36hfGf/xz69Kmevn592tE651x5eLIpoW7dQnXazJnw1FPwjW9Ay5ah2u3EE6FnT7j0UqiqSjtS55wrrUbdyCm9AAAXvUlEQVTbN5qkQ4BfErrBH2Vmz9U0/+DBg60cP+pcvBjuuQduvRWmT68uP/RQOOoo6N8fBgyAnXaCrQo96sw551ImabKZDS40X4PqQUDSHYSHIi00s90T5UMJD/JqDtxmZlcTngWReXrk3ByrS0WXLvD978PFF8NLL4WkM3o0PPtsGJK6datOPv37Vw/du0PnztCiQZ0951xT1qCubOKz7j8G7s4kG0nNCQ9QOoKQVF4DhgHvmtkmSdsDfzCzU2tad7mubHJZvhzGjIE33wxValVVMGsWrFtX83IdOsC2224+dOwYroi22gratQt/27cPiatnz9BCrl278uyXc67xa5RXNmb2QnyIV9J+QJWZzQKQNAo43szeidOXAa3LFuQW6NgRvvOdzcs2boS5c8P9nkwCygwffQRLl8KKFWGYNat229t2W+jaNdw/ygxdu8LAgWHo3x+23z4MnTpBM7+z55yrowaVbPLoDsxJvJ4LDJF0InAU0JHwyNPPkHQOcA5Ar169Shxm7TRvDr17h+Hwwz87fdOmcEW0ZMnmw4oVsHr15sPKlfDhh6FF3Ny51fNm+/vfc8fRtWtIPNttF4beveGLX4T99oPWraFVqzA451w+jSHZ5GRmY4AxBeYZCYyEUI1WjrjqS7Nm4b5N587hnk6xNm2CBQtg2bLQ9Hr9+lBd9+GHMGNGGGbPhoULw7B8ebiSqqk3hGbNYM894YAD4MADw99evUCq+3465xqHxpBs5gE9E697xDKXQ7Nm4f5Nt27Fzb92bWhBt2BBdQJ65x14/nl46y0wgzVrYMqUMNx4Y1iuWzfYe+/QmKF1a9htNxg0CPbYA9q0Kd3+OecqU2NINq8BAyT1JSSZU4BvphtS49G6dUgY3bvnn2f1apg0CV58MbSwe+ml0Bv2/BxPXW/ePPzItXXrcK+oRYswtG0LX/oS7Lor7LVX+PGrXxk513g0qGQj6QHgEKCLpLnAlWZ2u6QLgfGEps93mNnbKYbZ5Gy1FRx8cBggVNVVVcG0aeHe0McfwxtvwOuvh98WzZyZez3PPFM9PnAg3HdfuBrypONcw9egmj6XUppNn5uSTz6B99+HDRvC/aLM3/nz4cknQwOGiROrGzB07w6nnBKST6aFXIcO4crHq+OcS1+xTZ892USebCrHJ5+EZwQ99FD+h9K1aRNaxrVuHa6sPv/58Eyh1q1hm21CDwwDBoQqu7Ztyxu/c02JJ5ta8mRTecxCQ4Tx40MDhUwjhSVLQou5Ykjhh6y77AKHHBKS0h57hObbzrm682RTS55sGpYlS2DVqtBabtmycE9o6dLweunScM9o5syQlHI9S6hPH9h99/Dohx49Qu8KPXuG3w75b4acK16j7EHAuYxM9zwZ+++fe77168M9ookT4bXXQpdAL78cnjv03nufnb9Tp/Aj2l13Db8Z2n33UF3XvHkp9sK5psOvbCK/smk6PvkkdPEzfXpokDBnThieeSZcFWVr1iwknN13D63j9t47/O3f37vycc6r0WrJk41bvz4013777dBM+6WXwlXRokXh/lG2du1CAtpjjzDsuWf4jdA225Q/dufSUvHJRlJvYICZPSWpLdDCzFalEgyebFx+69eH7nwyvSRMmQKTJ4eyXAYMqL76GTQIdt453A/yqyDXGFV0spF0NqEDzM5mtpOkAcCfzeywsgcTebJxtbV0abgH9MYbYZg6NbzO9bjvjh3DfaVDD4V99gnJqHPn8sfsXH2r9GQzlfBogFfNbO9Y9qaZfb7swUSebFx9WLcu9B33+uuhC5+334Z33w1NtrP16QNHHBEeF77vvl795hqmSk82r5rZEElTzGxvSS2A181sj7IHE3mycaViFppgv/RSaAn3+uvhSmjNmup5mjWDL3whtIQ7+ujQBNu76XENQaUnm98Cy4HTgP8GzgfeMbMryh5M5MnGldOGDaHX7FGjYMKE0DBhw4bq6bvvDiedFJLPkCH+CHBXuSo92TQDhgNHAiJ0onmbpdg0zpONS9OKFfDcc/DUU/DXv4YWcBmdOoWqthNOgP/6L3+st6sslZ5stgI+NbON8XVzoLWZfVL2YCJPNq5SrFsXOiX9xz9CVz1VVdXTWrUKvWt/4Quh+50DDgj9wTmXlkpPNq8Ah5vZx/H11sCTZnZA2YOJPNm4SjVlCowZEx7b/cYbm//mp3VrOOggOOYY+Pa3N+9VwblyqPRkM9XM9ipUVk6ebFxDsGABvPACvPJKuPJ5553qae3awXHHwZFHhr+eeFw5FJts0vqZ2WpJgzIvJO0DrKlhfucc4Xk+J58M11wTmlUvWBAeMnfUUaEbnlGj4DvfCfMdfjjcdFP+H586V05pXdnsC4wCPiQ0ENgB+IaZTS57MJFf2biGbvp0ePppeOyx0M9bpnWbFO7xnHhiGPr2TTdO17hUdDUagKSWwMD4coaZ5fjddfl4snGNybJlIemMGRMaGXz6afW0vfeuTjy77ppejK5xaAjJ5gCgD4nHHJjZ3akEgycb13h9/DGMGwcPPwyPPx5eZ+yyCwwbFppVf/7z/kNSV3sVnWwk3QPsBEwFMo+2MjO7qOzBRJ5sXFPw6afhR6RjxsCjj4YroIxBg+CMM+ArXwld6ThXjEpPNtOBXdP8EWc2TzauqVm/PvyI9KGHQuJZsqR62kEHwZlnwte+Bu3bpxejq3yV3hrtLUKjAOdcSlq2DP2w3X57eIjcvfeGlm5t24bm1WeeCTvsAMOHhx6tnauLtK5sngX2AiYCazPlZnZc2YOJ/MrGuWDFCnjwQbjrLvjnP6vLjzkGvvtdOPZYv7fjqlV6NdrBucrN7Pl63MbngO8BXYCnzezmmub3ZOPcZ82cCTfeCDffHLrRgdBJ6MUXw6mnQps26cbn0lfR1WgxqbwHtIzjrwGvF1pO0h2SFkp6K6t8qKQZkqokXRq3Md3MzgO+DhxY7zvhXBMwYABcd12oZvvd70K12ltvwVlnhUYEv/pVmOZcIakkm/ikzoeAW2JRd+CRIha9Exiata7mwI3A0cCuwDBJu8ZpxwGPA0/US+DONVFdu8IPfwjvvw/33AN77RV6L/jpT6F379B0etw42LQp7UhdpUqrgcAFhKuNlQBmNhPYrtBCZvYCsDSreD+gysxmmdk6Qs8Ex8f5x5rZ0cCpudYn6RxJkyRNWpTs0905l1OrVvCtb4UHwD3xRGhQ0Lx5aM12zDHhtzq33hq6znEuKa1kszYmBgDikzq39OZRd2BO4vVcoLukQyTdIOkW8lzZmNlIMxtsZoO7du26hZt3rumRQku20aPhgw/gqqugZ8/QMeg558BOO8Gvfw2LF6cdqasUaSWb5yVdDrSVdATwIPBYfW7AzJ4zs4vM7Fwzu7E+1+2cq7bDDnD55aExwd13hx+HfvQRXHFFqGL70Y/8vo5LL9lcCiwC3gTOJVx5/GQL1zUP6Jl43SOWOefKqHXr8Eyd114L3eIMHRqq037/e9h5Z/jZzzb/4ahrWtJqjbbJzG41s5PN7KQ4vqXVaK8BAyT1ldQKOAUYW3/ROudqo1mzcP9m3DiYODH0QrBmDfzyl9CjB5x7LsyZU3g9rnFJqzXabEmzsocilnsAeBkYKGmupOFmtgG4EBgPTAdGm9nbpd0D51wx9t03dIfzf/8XrnQ+/RRGjoT+/UP1mt/TaTrS+lFn8hmCbYCTgc5m9rOyBxP5jzqdK71334URI+Cvfw2v27aFb34TLroI9tgj1dDcFqr0H3UuSQzzzOw64Ng0YnHOlc8uu4Snib72WmjNtmZN6Jttzz3D68ceg40bC6/HNTxpVaMNSgyDJZ1H4rk2zrnGbfDg8Dudd9+FCy8MVzj/+Accd1z4wei991Z3j+MahzQ74szYQOi65vdmNqPswURejeZcehYvhjvvhBtuqG480LVr6IPt/POhY8dUw3M1qOiOOCuRJxvn0rd2bfitzp/+BNOmhbK2beGCC+D734cdd0w3PvdZFZ1sJP2gpulm9odyxZLhyca5ymEGTz8NV18d/kJ4/s43vgGXXBKq2lxlqOgGAsBg4LuErma6A+cBg4D2cXDONWESHH54eJLoxIlw4omwYUO4l7P33nDUUZs/a8dVvrSubF4AjjWzVfF1e+BxMzuo7MFEfmXjXGWbNSv0RnDnnaEVG8DBB4euco44wh/olpZKv7LZHki2NVkXy5xzLqd+/eCmm0IDgh/9CNq1g+efD1c5gwaFp4tu2JB2lC6ftJLN3cBESSMkjQBeBe5KKRbnXAOy7bbw29+GZ+v87GehpdrUqfD1r8PAgeF3PP5bncqT1o86rwLOBJbF4Uwz+3UasTjnGqYuXeDnP4d580Lrtd69Q1XbsGHhCaM33lhd3ebSl9aVDUA7YKWZXQ/MldQ3xViccw1Uu3ahafSMGSHB9O0Ls2eHH4v26xd+u+NJJ31lSzaSdk+MXwn8GLgsFrUE7i1XLM65xqd16/AD0H/9C+6/H3bbLTxX53vfC71NX3yxd/yZpnJe2fSSdHUc/ypwHLAawMw+xJs8O+fqQYsWoSpt2rSQdD73OVi6FK6/PvRKcN55/lydNJQt2ZjZE0Cmm5p18fk1BiBpq3LF4ZxrGpo1C0nn7bdDB58DBoTyW24J93tOPTU8XdSVR1nv2ZjZ+Dg6WtItQEdJZwNPAbeWMxbnXNMgwZe/HKrXHnwQ+vQJ5fffH54g2rcvPPtsjatw9SCt1mi/Bx4CHgYGAj8zsz+mEYtzruk46aTQeOCRR2C77ULZe+/Bl74E22wTeixwpVH2ZCOpuaRnzWyCmf3IzH5oZhPKHYdzruk6/nhYsAD+8pfQ5xrAqlWhJ4LWrcPTRV39KnuyMbONwCZJHcq9beecSzrjjPDcnAceqO7uZt06OPnk8Pqee1INr1FJ63c2HwNvSrpd0g2ZIaVYnHNN3CmnwKZN8Oij0KpVdflpp4WkM3Jk6Inabbm0ks0Y4KfAC8DkxOCcc6k57rjwTJ2nnw7P0ck499zQuu0PfwhJydVeWXt9ltTLzD4o2wZrwXt9ds5le/XV0Hjgk082L//pT+HKK6F583TiqiSV2uvzI5kRSQ+XedvOOVcrQ4bA6tUweTL07Fld/stfhh+PXnQRrF+fXnwNSbmTTfKJE/1KuiGpX7wn5O1KnHN1MmgQfPBB6H+tV6/q8j/+MdzjueACWLkyvfgagnInG8szXhRJd0haKOmtrPKhkmZIqpJ0KYCZzTKz4XWM1znn/mPnncOjDebNgwMOqC6/6Sbo0AGOPBLefDO9+CpZuZPNnpJWSloF7BHHV0paJamY7wV3AkOTBZKaAzcCRwO7AsMk7VrfgTvnXMaOO8KLL8L8+XDssdXlEybAHnuEB7pN9iZPmyl3dzXNzWwbM2tvZi3ieOb1NkUs/wKwNKt4P6AqXsmsA0YBxxcTj6RzJE2SNGnRokW13h/nXNO2ww7w97+H+zoXXlhd/uSTMHhweMbOiy/6Iw4g3efZ1JfuwJzE67lAd0nbSvozsLeky3ItaGYjzWywmQ3u2rVrOWJ1zjVC7dqF+zcLF4bepTM++AC++MUw/YknYO7c9GJMW2NINjmZ2RIzO8/MdjKz/007Hudc49e1a2ihtmZN6PRzhx2qpx17bGjRdt11sGxZ02vF1hiSzTwg0SiRHrHMOedS0aZN6PRz/vzQuefnP1897fvfh86dwwPdli1LL8ZyawzJ5jVggKS+kloBpwBjU47JOecAOOyw8CC3mTOhf3/o2DGUL1wYks6QIfC1rzX++zoNKtlIegB4GRgoaa6k4Wa2AbgQGA9MB0ab2dtpxumcc9n69w8JZ9my8LucjIkTYcwYuO02ePnlMLz3XmphlkxZu6upZN5djXOuXDZuhKlTYcOG0PXNhKyHrEgwfToMHJhOfLVRbHc1LcoRjHPOuWrNm8M++4Txyy8PnX+uWxdez5gRrn4efzw8YyejffuGkXzy8SubyK9snHOV4NvfhnvvzT3t7rvD9EriVzbOOdcAnXUWzJoVrnYyPvoodJEzfXp6cdWVJxvnnKsgBx8ceh1Iuu660GR6yhS4777cy/XsCQcdVPr4tpQnG+ecq3AdOoS///hHGPKZNm3z3/RUEk82zjlX4b761XBVs2RJ7unPPBOq2j780JONc865LdSxI9xwQ/7pX/5yaL1WyV3gNKgfdTrnnPusVq3C30zz6UrkVzbOOdfAtWwZ/o4aBW+9VfO8uWy1FVxySf3GlM2TjXPONXCdO4e/Dz4YhtrafntPNs455wq44orQi3Tytzm1sfXW9RtPLp5snHOugevRIyScSuYNBJxzzpWcJxvnnHMl5x1xRpIWAe/Hlx2AFYnJydf5xrsAi+sYRvZ2t2S+fNNq2qfs101hH5OvK3Ef800vVFbMea2PfcwXS23n25J9zH7dFP5fK3kfe5tZ14JLm5kPWQMwMt/rGsYn1fd2t2S+fNNq2qemuI/J15W4j/mmFyor5rzWxz6W8lwWU9bU/l8rfR+LGbwaLbfHanidb7wU292S+fJNq2mfsl83hX1Mvq7Efcw3vVBZsee1PpTqXBZT1tT+Xyt9HwvyarR6ImmSFfFMh4bM97FxaAr7CE1jPxvSPvqVTf0ZmXYAZeD72Dg0hX2EprGfDWYf/crGOedcyfmVjXPOuZLzZOOcc67kPNk455wrOU82JSBpK0l3SbpV0qlpx1MKkvpJul3SQ2nHUkqSTojn8a+Sjkw7nlKQ9DlJf5b0kKTvph1PqcT35SRJX047llKRdIik/4vn85C040nyZFMkSXdIWijprazyoZJmSKqSdGksPhF4yMzOBo4re7BbqDb7aGazzGx4OpHWTS3385F4Hs8DvpFGvFuilvs43czOA74OHJhGvFuilu9JgB8Do8sbZd3Vcj8N+BhoA8wtd6w1quuvT5vKABwEDALeSpQ1B/4N9ANaAW8AuwKXAXvFee5PO/ZS7GNi+kNpx12m/bwGGJR27KXaR8KXonHAN9OOvRT7CBwBnAKcAXw57dhLuJ/N4vTtgfvSjj05+JVNkczsBWBpVvF+QJWFb/nrgFHA8YRvFD3iPA3mGNdyHxus2uyngt8A48zs9XLHuqVqey7NbKyZHQ00mGrfWu7jIcD+wDeBsyU1yvelmW2K05cBrcsYZkH+PJu66Q7MSbyeCwwBbgD+JOlY6r9riXLLuY+StgWuAvaWdJmZ/W8q0dWffOfyv4HDgQ6S+pvZn9MIrp7kO5eHEKp+WwNPpBBXfcq5j2Z2IYCkM4DFiQ/lhirfuTwROAroCPwpjcDy8WRTAma2Gjgz7ThKycyWEO5jNGpmdgPhy0OjZWbPAc+lHEZZmNmdacdQSmY2BhiTdhy5NJhLyQo1D+iZeN0jljUmTWEfoWnsp+9j49Hg9tOTTd28BgyQ1FdSK8INyLEpx1TfmsI+QtPYT9/HxqPB7acnmyJJegB4GRgoaa6k4Wa2AbgQGA9MB0ab2dtpxlkXTWEfoWnsp+9j49hHaDz76R1xOuecKzm/snHOOVdynmycc86VnCcb55xzJefJxjnnXMl5snHOOVdynmycc86VnHdX41wKJG0E3iS8B2cD3zaz5elG5Vzp+JWNc+lYY2Z7mdnuhB59L0g7IOdKyZONc+l7mdCLL5K2lvS0pNclvSnp+FjeR9L0+NTQtyU9KaltnLavpGmSpkr6XfZDtpyrBJ5snEuRpObAYVT3a/Up8FUzGwQcClwjSXHaAOBGM9sNWA58LZb/BTjXzPYCNpYteOdqwZONc+loK2kq8BHhqYoTYrmAX0uaBjxFuOLZPk6bbWZT4/hkoI+kjkB7M3s5lt9fluidqyVPNs6lY028EulNSDCZezanAl2BfeL0BYTnyQOsTSy/EW/g4xoQTzbOpcjMPgEuAi6R1ALoACw0s/WSDiUko5qWXw6skjQkFp1S0oCd20L+zci5lJnZlFhtNgy4D3hM0pvAJODdIlYxHLhV0ibgeWBFyYJ1bgv5Iwaca+AkbW1mH8fxS4FuZva9lMNybjN+ZeNcw3espMsI7+f3gTPSDce5z/IrG+eccyXnDQScc86VnCcb55xzJefJxjnnXMl5snHOOVdynmycc86VnCcb55xzJff/ozCIx0QQbPcAAAAASUVORK5CYII=\n",
      "text/plain": [
       "<Figure size 432x288 with 2 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    }
   ],
   "source": [
    "# Set redo_tokenization to True if you want to do again the tokenization and the dictionnary, \n",
    "# Otherwise, the saved dictionnary will be loaded\n",
    "from q1_to_5_CS276 import *\n",
    "\n",
    "analysis(path_to_cs276_dir, redo_tokenization=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Index booléen\n",
    "La classe BooleanIndex hérite de Index et permet de créer un index inverser sur la base de données CACM ou CS276 selon la méthode appelée. Les requêtes booléennes sont traitées comme un arbre d'opérations en donnant la priorité à NOT, puis OR, puis AND. On peut aussi entrer des requêtes parenthésées. Les fonctions utiles à la gestion de l'arbre sont dans tree_operator.py et permettent de transformer une requête sous forme de chaîne de caractère parenthésée à un arbre d'opérateurs où chaque nœud est une opération booléenne et chaque feuille est une token_id. L'arbre est ensuite parcouru par l'index et chaque opération est calculée.\n",
    "\n",
    "Pour profiter au mieux de la structure de sparse matrix utilisée pour l'index inversée, l'opérations NOT n'est pas calculée telle quelle. Le résultat renvoyé par le parcours d'arbre renvoie une liste sparse avec un booléen par document, et un booléen indiquant si les documents renvoyés sont les documents pertinents ou au contraire les documents inintéressants. Cela permet de garder une quantité restreinte de documents en mémoire même lorsqu'on demande une requête du type \"NOT paper\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from tree_operator import *\n",
    "from boolean_index import BooleanIndex"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Pertinence des requêtes\n",
    "Les fonctions de cette cellule implémentent toutes les mesures de pertinence des résultats et permettent de faire des rapport facilement pour l'évaluation des résultats de l'index sur les requêtes qu'on lui soumet"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "from metrics import *\n",
    "import re"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Index booléen sur CACM\n",
    "L'index est créé en une vingtaine de secondes et permet de traiter les requêtes en un temps de l'ordre du centième de seconde. Cependant, la pertinence des résultats est très basse. Notamment, le champ \"auteur\" n'étant pas utilisé pour le référencement, il est impossible de lancer des recherches sur un nom d'auteur."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Building index for CACM...\n",
      "Index built in 20.5s\n"
     ]
    }
   ],
   "source": [
    "index = BooleanIndex()\n",
    "start = time()\n",
    "print(\"Building index for CACM...\")\n",
    "index.build_cacm(path_to_cacm_file)\n",
    "print(\"Index built in {:.1f}s\".format(time()-start))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "scrolled": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['tss', 'or', 'time', 'and', 'share', 'and', 'system']\n",
      "or\n",
      "\ttss\n",
      "\tand\n",
      "\t\ttime\n",
      "\t\tand\n",
      "\t\t\tshare\n",
      "\t\t\tsystem\n",
      "Query treated in 0.1s\n",
      "Docs found for query : TSS or (time and sharing and system)\n",
      "[ 322  585  971  972 1071 1170 1341 1349 1391 1410 1487 1506 1519 1523\n",
      " 1526 1544 1571 1572 1605 1657 1677 1740 1748 1749 1752 1792 1805 1811\n",
      " 1827 1844 1877 1892 1901 1908 1938 1951 1960 1978 2036 2054 2069 2092\n",
      " 2128 2130 2151 2218 2219 2228 2320 2344 2357 2371 2375 2377 2378 2380\n",
      " 2399 2424 2439 2499 2500 2501 2522 2535 2621 2626 2629 2667 2730 2838\n",
      " 2867 2920 2938 2947 2948 2950 2951 2954 3025 3112 3174 3204]\n",
      "Docs we should have found :\n",
      "[1410 1572 1605 2020 2358]\n",
      "precision :\t\t\t0.04\n",
      "recall :\t\t\t0.60\n",
      "e_measure :\t\t\t0.96\n",
      "f_measure :\t\t\t0.07\n",
      "r_measure :\t\t\t0.00\n",
      "interpolated_precisions :\t[0.15789473684210525, 0.15789473684210525, 0.15789473684210525, 0.15789473684210525, 0.15789473684210525, 0.15789473684210525, 0.15789473684210525, 0, 0, 0, 0]\n",
      "average_precision :\t\t0.10\n"
     ]
    },
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAYsAAAEWCAYAAACXGLsWAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAHKRJREFUeJzt3X+UXWV97/H3h8TwQwKBMHolP0gwoe0gCDgEu66CCsUEC0PbAAlYSEWiYtpegd7GtitAkPZSr6CWeCWVyO8GzBLvVGIDCpYuBJrhh8AEU4cIZAK3jBB+hB9C4Hv/2M/Qw+HkPCdh9pkzmc9rrVnZ+9nP3uf7TDL5zN7POXsrIjAzM6tnh6EuwMzMWp/DwszMshwWZmaW5bAwM7Msh4WZmWU5LMzMLMthYdsVST2SPpbpM1nSJkmjmlTWoJM0RVJIGp3Wfyrps0Ndl22/Rg91ATYySHoUeC/wOvAi8CNgQURsGszXiYj9G+jzOLDrYL6u2fbOZxbWTMdGxK7AIUAH8DfVHVTYbv5dDvzmvz3YnsZiW2+7+aG04SMiNlCcWXwA3ryEcqGkO4CXgH0l7S7pcklPStog6SuVl40knSHpYUkvSFoj6ZDU/qiko9LyDEndkp6X9J+SLk7t1Zdw9pbUJekZSb2Szqh4nfMk3SDpqvRaPZI6tjS21H+FpGskPQ/Mk7SDpIWSHpH0dDrenhX7fETSzyQ9K2m9pHmp/VOS7kv1r5d03rZ8vyWNkvRX6fVfkHSPpEnV34eKv4vPpuV5ku6QdImkp4ELUo0fqOjfJullSe9J678v6f7U72eSDtyWmq31OCys6SRNAo4B7qto/mNgPjAWeAy4AtgMTAMOBo4GBv4TOwE4DzgV2A04Dni6xkt9A/hGROwGvB+4YQslLQf6gL2B2cDfSvpExfbjUp9xQBdwaWaIncCK1P9a4E+B44Ej0mtsBJaksexDEZz/ALQBBwH3p+O8mMY4DvgU8AVJx2deu5azgLkU3/PdgM9QhHIjDgPWUVxCXAx8Px1rwInAv0bEU5IOBpYBnwPGA5cBXZJ23IaardVEhL/8VfoX8CiwCXiWIgy+Beyctv0UWFzR973Abwa2p7a5wG1peRXw53Ve56i0fDtwPrBXVZ8pQFDM2U2imEcZW7H974Ar0vJ5wI8rtrUDL9cZ53nA7VVtDwNHVqy/D3gtvf6XgRsb/B5+HbikegwV38PPbmG/tUBnjfa3HKP6OMA84PGqfY4CHqlYvwM4NS3/H+CCGq99xFD/+/PXO//ymYU10/ERMS4i9omIMyPi5Ypt6yuW9wHeBTyZLmc8S/Fb6nvS9knAIw283unAfsAvJK2W9Ps1+uwNPBMRL1S0PQZMqFj/fxXLLwE7SRot6ZT0rqpNkn60hbEMjOfGirE8TBFQ7603FkmHSbpNUr+k54DPA3tlR/12jX6/aqkey23ALqm2KRRnQjembfsAZw+MM411EsX32IY5T1hZq6i8/fF6ijOLvSJic42+6ykuK9U/YMQvgblpwvwPgRWSxld1ewLYU9LYisCYDGxo4PjXUlxmetumGvV+JiLuqO4oaT0wYwsvcR3FJa9ZEfGKpK+zbWEx8P16qKr9xfTnLsDzafm/VfV5y1gi4nVJN1Cc6f0n8MOK79t64MKIuHAbarQW5zMLazkR8SRwM/A1SbulCeL3SzoidfkOcI6kD6V3T01L1/7fQtKnJbVFxBsUl78A3qh6rfXAz4C/k7RTmpA9HbhmEIf0beDCgRrTpHBn2nYtcJSkE9PZynhJB6VtYynOel6RNAM4eRtf/zsUk9PT0/frQEnjI6KfIhQ/nSbBP0MDIUwRYicBp6TlAf8IfD6ddUjSu9Mk/dhtrNtaiMPCWtWpwBhgDcWE8AqKa/1ExPeACyn+o3oB+AGwZ41jzAR6JG2imOyeU3Xpa8Bciuv3T1BcUjk3In48iGP5BsXE+M2SXgDuopg4JorPfBwDnA08QzG5/cG035nA4rTPIrY8QZ9zcdr3ZooziMuBndO2M4C/oHiDwP4UwVlXRNxNcVayN8Xk/EB7dzrepRR/Z70U8x62HVCEH35kZmb1+czCzMyyHBZmZpblsDAzsyyHhZmZZW03n7PYa6+9YsqUKUNdhpnZsHLPPff8OiLacv22m7CYMmUK3d3dQ12GmdmwIumxRvr5MpSZmWU5LMzMLKvUsJA0U9JaFc8IWFhj++GS7pW0WdLsqm2TJd2s4pkFa9JNy8zMbAiUFhYqHlSzBJhFcVvnuZLaq7o9TnE7gOt4u6uAr0bE71DcaO2psmo1M7P6ypzgngH0RsQ6AEnLKR4Ks2agQ0Q8mra95eZuKVRGR8Qtqd+gPqfZzMy2TpmXoSbw1nvh9/HWZwTUsx/wrKTvp8dKflUVj9Q0M7PmatUJ7tHAR4FzgEOBfalx90pJ81U8Y7m7v7+/uRWamY0gZV6G2kDxlKwBE2nggTJJH3B/xSWsHwAfpri18psiYimwFKCjo2Obb597/j/3sOaJ5/MdzaxhnQdN4OTDJg91GTZIyjyzWA1MlzRV0hhgDsU9/Rvdd5ykgU8VfoKKuQ4za21rnnye/3t/o78b2nBQ2plFRGyWtABYBYwClkVEj6TFQHdEdEk6lOJhM3sAx0o6PyL2T49uPAf4iSQB91A8hasU5x67f1mHNhuRTrrszqEuwQZZqbf7iIiVwMqqtkUVy6spLk/V2vcW4MAy6zMzs8a06gS3mZm1EIeFmZllOSzMzCzLYWFmZlkOCzMzy3JYmJlZlsPCzMyyHBZmZpblsDAzsyyHhZmZZTkszMwsy2FhZmZZDgszM8tyWJiZWZbDwszMshwWZmaWVWpYSJopaa2kXkkLa2w/XNK9kjZLml1j+26S+iRdWmadZmZWX2lhIWkUsASYBbQDcyW1V3V7HJgHXLeFw1wA3F5WjWZm1pgyzyxmAL0RsS4iXgWWA52VHSLi0Yh4AHijemdJHwLeC9xcYo1mZtaAMsNiArC+Yr0vtWVJ2gH4GnBOCXWZmdlWatUJ7jOBlRHRV6+TpPmSuiV19/f3N6k0M7ORZ3SJx94ATKpYn5jaGvG7wEclnQnsCoyRtCki3jJJHhFLgaUAHR0d8c5LNjOzWsoMi9XAdElTKUJiDnByIztGxCkDy5LmAR3VQWFmZs1T2mWoiNgMLABWAQ8DN0REj6TFko4DkHSopD7gBOAyST1l1WNmZtuuzDMLImIlsLKqbVHF8mqKy1P1jnEFcEUJ5ZmZWYNadYLbzMxaiMPCzMyyHBZmZpblsDAzsyyHhZmZZTkszMwsy2FhZmZZDgszM8tyWJiZWZbDwszMshwWZmaW5bAwM7Msh4WZmWU5LMzMLMthYWZmWQ4LMzPLcliYmVlWqWEhaaaktZJ6Jb3tGdqSDpd0r6TNkmZXtB8k6U5JPZIekHRSmXWamVl9pYWFpFHAEmAW0A7MldRe1e1xYB5wXVX7S8CpEbE/MBP4uqRxZdVqZmb1lfkM7hlAb0SsA5C0HOgE1gx0iIhH07Y3KneMiP+oWH5C0lNAG/BsifWamdkWlHkZagKwvmK9L7VtFUkzgDHAIzW2zZfULam7v79/mws1M7P6WnqCW9L7gKuBP4mIN6q3R8TSiOiIiI62trbmF2hmNkKUGRYbgEkV6xNTW0Mk7QbcBPx1RNw1yLWZmdlWKDMsVgPTJU2VNAaYA3Q1smPqfyNwVUSsKLFGMzNrQGlhERGbgQXAKuBh4IaI6JG0WNJxAJIOldQHnABcJqkn7X4icDgwT9L96eugsmo1M7P6ynw3FBGxElhZ1baoYnk1xeWp6v2uAa4pszYzM2tcS09wm5lZa3BYmJlZlsPCzMyyHBZmZpblsDAzsyyHhZmZZTkszMwsy2FhZmZZDgszM8tyWJiZWZbDwszMshwWZmaW5bAwM7Msh4WZmWU5LMzMLMthYWZmWaWGhaSZktZK6pW0sMb2wyXdK2mzpNlV206T9Mv0dVqZdZqZWX2lhYWkUcASYBbQDsyV1F7V7XFgHnBd1b57AucChwEzgHMl7VFWrWZmVl+ZZxYzgN6IWBcRrwLLgc7KDhHxaEQ8ALxRte8ngVsi4pmI2AjcAswssVYzM6ujzLCYAKyvWO9LbYO2r6T5kroldff3929zoWZmVt+wnuCOiKUR0RERHW1tbUNdjpnZdqvMsNgATKpYn5jayt7XzMwGWZlhsRqYLmmqpDHAHKCrwX1XAUdL2iNNbB+d2szMbAiUFhYRsRlYQPGf/MPADRHRI2mxpOMAJB0qqQ84AbhMUk/a9xngAorAWQ0sTm1mZjYERpd58IhYCaysaltUsbya4hJTrX2XAcvKrM/MzBozrCe4zcysORwWZmaW5bAwM7Msh4WZmWU5LMzMLMthYWZmWQ4LMzPLcliYmVlW3Q/lSTqr3vaIuHhwyzEzs1aU+wT32KZUYWZmLa1uWETE+c0qxMzMWlfuMtQ3622PiD8b3HLMzKwV5S5D3dOUKszMrKXlLkNd2axCzMysdTV0i3JJbcBfAu3ATgPtEfGJkuoyM7MW0ujnLK6leIDRVOB84FGKhxKZmdkI0GhYjI+Iy4HXIuJfI+IzQPasQtJMSWsl9UpaWGP7jpKuT9vvljQltb9L0pWSHpT0sKQvb8WYzMxskDUaFq+lP5+U9ClJBwN71ttB0ihgCTCL4vLVXEntVd1OBzZGxDTgEuCi1H4CsGNEHAB8CPjcQJCYmVnzNRoWX5G0O3A2cA7wHeBLmX1mAL0RsS4iXgWWA51VfTqBgUn0FcCRkgQE8G5Jo4GdgVeB5xus1czMBllDE9wR8cO0+Bzw8QaPPQFYX7HeBxy2pT4RsVnSc8B4iuDoBJ4EdgG+FBHPVL+ApPnAfIDJkyc3WJaZmW2ths4s0vzBuIr1PSQtK68sZgCvA3tTTKqfLWnf6k4RsTQiOiKio62trcRyzMxGtkYvQx0YEc8OrETERuDgzD4bgEkV6xNTW80+6ZLT7sDTwMnAv0TEaxHxFHAH0NFgrWZmNsgaDYsdJO0xsCJpT/KXsFYD0yVNlTQGmAN0VfXpAk5Ly7OBWyMigMdJ77aS9G7gw8AvGqzVzMwGWUNzFsDXgDslfS+tnwBcWG+HNAexAFgFjAKWRUSPpMVAd0R0AZcDV0vqBZ6hCBQo3kX1XUk9gIDvRsQDWzMwMzMbPI1OcF8lqZv/+mzFH0bEmgb2WwmsrGpbVLH8CkXwVO+3qVa7mZkNja15Ut6ewIsRcSnQL2lqSTWZmVmLafTdUOdS3Btq4JPU7wKuKasoMzNrLY2eWfwBcBzwIkBEPIGfomdmNmI0GhavpncpBbz5DiUzMxshGg2LGyRdBoyTdAbwY4pbfpiZ2QjQ6Luh/rek36O4P9NvAYsi4pZSKzMzs5bR6OcsSOFwC4CkHSSdEhHXllaZmZm1jLqXoSTtJunLki6VdLQKC4B1wInNKdHMzIZa7sziamAjcCfwWeCvKD5RfXxE3F9ybWZm1iJyYbFvegARkr5DccvwyemT12ZmNkLk3g018IQ8IuJ1oM9BYWY28uTOLD4oaeAJdQJ2TusCIiJ2K7U6MzNrCXXDIiJGNasQMzNrXVtzI0EzMxuhHBZmZpblsDAzs6xSw0LSTElrJfVKWlhj+46Srk/b75Y0pWLbgZLulNQj6UFJO5VZq5mZbVlpYSFpFMXjUWcB7cBcSe1V3U4HNkbENOAS4KK072iK52V8PiL2Bz5Gxdt4zcysuco8s5gB9EbEuoh4FVgOdFb16QSuTMsrgCMlCTgaeCAifg4QEU+nz3mYmdkQKDMsJgDrK9b7UlvNPhGxGXgOGA/sB4SkVZLulfQ/a72ApPmSuiV19/f3D/oAzMys0KoT3KOBjwCnpD//QNKR1Z0iYmlEdERER1tbW7NrNDMbMcoMiw3ApIr1iamtZp80T7E78DTFWcjtEfHriHgJWAkcUmKtZmZWR5lhsRqYLmmqpDHAHKCrqk8XcFpang3cmh7fugo4QNIuKUSOANaUWKuZmdXR8MOPtlZEbE7PvlgFjAKWRUSPpMVAd0R0AZcDV0vqBZ6hCBQiYqOkiykCJ4CVEXFTWbWamVl9pYUFQESspLiEVNm2qGL5FeCELex7DcXbZ83MbIi16gS3mZm1EIeFmZllOSzMzCzLYWFmZlkOCzMzy3JYmJlZlsPCzMyyHBZmZpblsDAzsyyHhZmZZTkszMwsy2FhZmZZDgszM8tyWJiZWZbDwszMshwWZmaWVWpYSJopaa2kXkkLa2zfUdL1afvdkqZUbZ8saZOkc8qs08zM6istLCSNApYAs4B2YK6k9qpupwMbI2IacAlwUdX2i4EflVWjmZk1pswzixlAb0Ssi4hXgeVAZ1WfTuDKtLwCOFKSACQdD/wK6CmxRjMza0CZYTEBWF+x3pfaavaJiM3Ac8B4SbsCfwmcX+8FJM2X1C2pu7+/f9AKNzOzt2rVCe7zgEsiYlO9ThGxNCI6IqKjra2tOZWZmY1Ao0s89gZgUsX6xNRWq0+fpNHA7sDTwGHAbEl/D4wD3pD0SkRcWmK9Zma2BWWGxWpguqSpFKEwBzi5qk8XcBpwJzAbuDUiAvjoQAdJ5wGbHBRmZkOntLCIiM2SFgCrgFHAsojokbQY6I6ILuBy4GpJvcAzFIFiZmYtpswzCyJiJbCyqm1RxfIrwAmZY5xXSnFmZtawVp3gNjOzFuKwMDOzLIeFmZllOSzMzCzLYWFmZlkOCzMzy3JYmJlZlsPCzMyyHBZmZpblsDAzsyyHhZmZZTkszMwsy2FhZmZZDgszM8tyWJiZWZbDwszMskoNC0kzJa2V1CtpYY3tO0q6Pm2/W9KU1P57ku6R9GD68xNl1mlmZvWVFhaSRgFLgFlAOzBXUntVt9OBjRExDbgEuCi1/xo4NiIOoHhG99Vl1WlmZnllnlnMAHojYl1EvAosBzqr+nQCV6blFcCRkhQR90XEE6m9B9hZ0o4l1mpmZnWUGRYTgPUV632prWafiNgMPAeMr+rzR8C9EfGb6heQNF9St6Tu/v7+QSvczMzeqqUnuCXtT3Fp6nO1tkfE0ojoiIiOtra25hZnZjaClBkWG4BJFesTU1vNPpJGA7sDT6f1icCNwKkR8UiJdZqZWUaZYbEamC5pqqQxwBygq6pPF8UENsBs4NaICEnjgJuAhRFxR4k1mplZA0oLizQHsQBYBTwM3BARPZIWSzoudbscGC+pFzgLGHh77QJgGrBI0v3p6z1l1WpmZvWNLvPgEbESWFnVtqhi+RXghBr7fQX4Spm1mZlZ41p6gtvMzFqDw8LMzLIcFmZmluWwMDOzLIeFmZllOSzMzCzLYWFmZlkOCzMzy3JYmJlZlsPCzMyyHBZmZpblsDAzsyyHhZmZZTkszMwsy2FhZmZZDgszM8sqNSwkzZS0VlKvpIU1tu8o6fq0/W5JUyq2fTm1r5X0yTLrNDOz+koLC0mjgCXALKAdmCupvarb6cDGiJgGXAJclPZtp3hm9/7ATOBb6XhmZjYEynys6gygNyLWAUhaDnQCayr6dALnpeUVwKWSlNqXR8RvgF+lZ3TPAO4ssV4zG0Rrnnyeky7zj2wztO+9G+ceu3+pr1FmWEwA1les9wGHbalPRGyW9BwwPrXfVbXvhOoXkDQfmA8wefLkQSvczN6ZzoPe9uNqw1yZYVG6iFgKLAXo6OiIIS7HzJKTD5vMyYf5F7jtSZkT3BuASRXrE1NbzT6SRgO7A083uK+ZmTVJmWGxGpguaaqkMRQT1l1VfbqA09LybODWiIjUPie9W2oqMB349xJrNTOzOkq7DJXmIBYAq4BRwLKI6JG0GOiOiC7gcuDqNIH9DEWgkPrdQDEZvhn4YkS8XlatZmZWn4pf5Ie/jo6O6O7uHuoyzMyGFUn3RERHrp8/wW1mZlkOCzMzy3JYmJlZlsPCzMyytpsJbkn9wGPv4BB7Ab8epHKGi5E25pE2XvCYR4p3MuZ9IqIt12m7CYt3SlJ3I+8I2J6MtDGPtPGCxzxSNGPMvgxlZmZZDgszM8tyWPyXpUNdwBAYaWMeaeMFj3mkKH3MnrMwM7Msn1mYmVmWw8LMzLJGVFhImilpraReSQtrbN9R0vVp+92SpjS/ysHVwJjPkrRG0gOSfiJpn6GoczDlxlzR748khaRh/zbLRsYs6cT0d90j6bpm1zjYGvi3PVnSbZLuS/++jxmKOgeLpGWSnpL00Ba2S9I30/fjAUmHDGoBETEivihuk/4IsC8wBvg50F7V50zg22l5DnD9UNfdhDF/HNglLX9hJIw59RsL3E7x+N6Ooa67CX/P04H7gD3S+nuGuu4mjHkp8IW03A48OtR1v8MxHw4cAjy0he3HAD8CBHwYuHswX38knVnMAHojYl1EvAosBzqr+nQCV6blFcCRktTEGgdbdswRcVtEvJRW76J4KuFw1sjfM8AFwEXAK80sriSNjPkMYElEbASIiKeaXONga2TMAeyWlncHnmhifYMuIm6neO7PlnQCV0XhLmCcpPcN1uuPpLCYAKyvWO9LbTX7RMRm4DlgfFOqK0cjY650OsVvJsNZdszp9HxSRNzUzMJK1Mjf837AfpLukHSXpJlNq64cjYz5PODTkvqAlcCfNqe0IbO1P+9bpbQn5dnwIunTQAdwxFDXUiZJOwAXA/OGuJRmG01xKepjFGePt0s6ICKeHdKqyjUXuCIivibpdymeyvmBiHhjqAsbjkbSmcUGYFLF+sTUVrOPpNEUp65PN6W6cjQyZiQdBfw1cFxE/KZJtZUlN+axwAeAn0p6lOLabtcwn+Ru5O+5D+iKiNci4lfAf1CEx3DVyJhPB24AiIg7gZ0obri3vWro531bjaSwWA1MlzRV0hiKCeyuqj5dwGlpeTZwa6SZo2EqO2ZJBwOXUQTFcL+ODZkxR8RzEbFXREyJiCkU8zTHRcRwfiZvI/+2f0BxVoGkvSguS61rZpGDrJExPw4cCSDpdyjCor+pVTZXF3BqelfUh4HnIuLJwTr4iLkMFRGbJS0AVlG8k2JZRPRIWgx0R0QXcDnFqWovxUTSnKGr+J1rcMxfBXYFvpfm8h+PiOOGrOh3qMExb1caHPMq4GhJa4DXgb+IiGF71tzgmM8G/lHSlygmu+cN51/+JP0TReDvleZhzgXeBRAR36aYlzkG6AVeAv5kUF9/GH/vzMysSUbSZSgzM9tGDgszM8tyWJiZWZbDwszMshwWZmaW5bAwqyDpdUn3S3pI0vck7TIIx+yQ9M062/eWtOKdvo5ZmfzWWbMKkjZFxK5p+Vrgnoi4uGK7KH5ufMsIG1F8ZmG2Zf8GTJM0JT034SrgIWCSpKMl3Snp3nQGMhAwh0r6maSfS/p3SWMlfUzSD9P2I9KZy/3pOQtj0/EfStt3kvRdSQ+m7R9P7fMkfV/Sv0j6paS/H6LviY1QDguzGtK9wWYBD6am6cC3ImJ/4EXgb4CjIuIQoBs4K9124nrgzyPig8BRwMtVhz4H+GJEHAR8tMb2LwIREQdQ3AjvSkk7pW0HAScBBwAnSZqEWZM4LMzeamdJ91MEwOMUt4ABeCw9IwCKmw+2A3ekvqcB+wC/BTwZEasBIuL5dKv7SncAF0v6M2Bcje0fAa5J+/8CeIziPk4AP0n3tnoFWJNe06wpRsy9ocwa9HL6rf9N6Z5ZL1Y2AbdExNyqfgfkDh4R/0vSTRT38LlD0idp/AFMlXcEfh3//FoT+czCbOvdBfx3SdMAJL1b0n7AWuB9kg5N7WPT5aw3SXp/RDwYERdR3Dn1t6uO/W/AKanvfsDkdFyzIeWwMNtKEdFP8fCkf5L0AHAn8Nvp8Z4nAf8g6efALRS3xa70P9Lbch8AXuPtTyb8FrCDpAcp5j/mbQfPGLHtgN86a2ZmWT6zMDOzLIeFmZllOSzMzCzLYWFmZlkOCzMzy3JYmJlZlsPCzMyy/j9ReEzrktCKVAAAAABJRU5ErkJggg==\n",
      "text/plain": [
       "<Figure size 432x288 with 1 Axes>"
      ]
     },
     "metadata": {
      "needs_background": "light"
     },
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "query 1 :\tr : 0.60\tp : 0.04\tf1 : 0.07\t(0.01s)\n",
      "Warning : word \"prieve\" not in the index\n",
      "Warning : word \"udo\" not in the index\n",
      "Warning : word \"pooch\" not in the index\n",
      "query 2 :\tr : 0.00\tp : NaN\t\tf1 : 0.00\t(0.01s)\n",
      "query 3 :\tr : 0.00\tp : NaN\t\tf1 : 0.00\t(0.01s)\n",
      "query 4 :\tr : 1.00\tp : 0.00\tf1 : 0.01\t(0.03s)\n",
      "query 5 :\tr : 0.25\tp : 0.67\tf1 : 0.36\t(0.01s)\n",
      "query 6 :\tr : 0.33\tp : 0.00\tf1 : 0.00\t(0.01s)\n",
      "query 7 :\tr : 0.00\tp : 0.00\tf1 : 0.00\t(0.00s)\n",
      "query 8 :\tr : 0.00\tp : NaN\t\tf1 : 0.00\t(0.01s)\n",
      "query 9 :\tr : 0.00\tp : NaN\t\tf1 : 0.00\t(0.01s)\n",
      "query 10 :\tr : 0.23\tp : 0.53\tf1 : 0.32\t(0.00s)\n",
      "Warning : word \"very\" is actually a stopword\n",
      "query 11 :\tr : 0.58\tp : 0.31\tf1 : 0.40\t(0.01s)\n",
      "query 12 :\tr : 0.20\tp : 1.00\tf1 : 0.33\t(0.00s)\n",
      "query 13 :\tr : 0.00\tp : NaN\t\tf1 : 0.00\t(0.00s)\n",
      "query 14 :\tr : 0.00\tp : NaN\t\tf1 : 0.00\t(0.01s)\n",
      "query 15 :\tr : 0.00\tp : NaN\t\tf1 : 0.00\t(0.00s)\n",
      "query 16 :\tr : 0.00\tp : NaN\t\tf1 : 0.00\t(0.01s)\n",
      "query 17 :\tr : 0.06\tp : 1.00\tf1 : 0.12\t(0.00s)\n",
      "query 18 :\tr : 0.00\tp : NaN\t\tf1 : 0.00\t(0.01s)\n",
      "query 19 :\tr : 0.73\tp : 0.32\tf1 : 0.44\t(0.00s)\n",
      "query 20 :\tr : 0.33\tp : 1.00\tf1 : 0.50\t(0.01s)\n"
     ]
    }
   ],
   "source": [
    "queries = [\n",
    "    \"\", # query id starts at 1 in the file\n",
    "    #  What articles exist which deal with TSS (Time Sharing System), an\n",
    "    # operating system for IBM computers?\n",
    "    \"TSS or (time and sharing and system)\",\n",
    "\n",
    "    \"Prieve or (Udo and Pooch)\",\n",
    "    \"intermediate and languages and used and construction and multi-targeted and compilers\",\n",
    "    \"(mechanisms and communicating and disjoint and processes) and not abstract\",\n",
    "    \"(design or implementation) and ((editing and interfaces) or window-managers or (command and interprets))\",\n",
    "    \"(robotics or (motion and planning) or (geometric or combinatorial)) and not (dynamics and arm and motion)\",\n",
    "    \"(distributed and algorithm) and (fault-tolerance or correctness)\",\n",
    "    \"resource and addressing and network and operating and system\",\n",
    "    \"security and local and (networks or (network and operating and system) or (distributed and system)\",\n",
    "    \"parallel and language\",\n",
    "    \"SETL or (very and high and level and language)\",\n",
    "    \"portable and operating and system\",\n",
    "    \"code and optimization and space and efficiency\",\n",
    "    \"optimal and implementation and sort and algorithms and database and management and applications\",\n",
    "    \"horizontal and microcode and optimization and loops\",\n",
    "    \"file and handling and operating and systems and multiple and processes and message and passing\",\n",
    "    \"optimization and intermediate and machine and code\",\n",
    "    \"languages and compilers and parallel and processors and microcoded and machines\",\n",
    "    \"parallel and algorithms\",\n",
    "    \"graph and theoretic and algorithms and sparse and matrices\"\n",
    "    ]\n",
    "truth_queries = read_truth(path_to_qrels)\n",
    "\n",
    "full_query_report(index, queries[1], np.array(truth_queries[1]))\n",
    "\n",
    "\n",
    "result_queries = []\n",
    "\n",
    "for id_query in range(1, len(queries)):\n",
    "    result = short_query_report(index, id_query, queries[id_query], truth_queries[id_query])\n",
    "    result_queries += [result]\n",
    "    \n",
    "# mean_average_precision = compute_mean_average_precision(result_queries, truth_queries)\n",
    "# print(\"mean average precision on {} queries : {}\".format(id_query, mean_average_precision))\n",
    "\n",
    "\n",
    "    \n",
    "# mean_average_precision = compute_mean_average_precision(docs, truth_query)\n",
    "# print(\"mean_average_precision : {:.2f}\".format(mean_average_precision))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Index vectoriel sur CACM\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "from vectorized_index import *"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Construction time = 21.3\n"
     ]
    }
   ],
   "source": [
    "from time import time\n",
    "begin = time()\n",
    "\n",
    "vec_index = VectorizedIndex()\n",
    "vec_index.build_cacm(path_to_cacm_file)\n",
    "print('Construction time = {:.1f}'.format(time() - begin))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "query 1 :\tr : 0.40\tp : 0.20\tf1 : 0.27\t(0.37s)\n",
      "query 2 :\tr : 0.00\tp : 0.00\tf1 : 0.00\t(0.24s)\n",
      "query 3 :\tr : 0.17\tp : 0.10\tf1 : 0.12\t(0.24s)\n",
      "query 4 :\tr : 0.08\tp : 0.10\tf1 : 0.09\t(0.25s)\n",
      "query 5 :\tr : 0.25\tp : 0.20\tf1 : 0.22\t(0.25s)\n",
      "query 6 :\tr : 1.00\tp : 0.30\tf1 : 0.46\t(0.24s)\n",
      "query 7 :\tr : 0.18\tp : 0.50\tf1 : 0.26\t(0.25s)\n",
      "query 8 :\tr : 0.33\tp : 0.10\tf1 : 0.15\t(0.24s)\n",
      "query 9 :\tr : 0.22\tp : 0.20\tf1 : 0.21\t(0.24s)\n",
      "query 10 :\tr : 0.26\tp : 0.90\tf1 : 0.40\t(0.25s)\n",
      "query 11 :\tr : 0.32\tp : 0.60\tf1 : 0.41\t(0.25s)\n",
      "query 12 :\tr : 0.40\tp : 0.20\tf1 : 0.27\t(0.24s)\n",
      "query 13 :\tr : 0.45\tp : 0.50\tf1 : 0.48\t(0.24s)\n",
      "query 14 :\tr : 0.11\tp : 0.50\tf1 : 0.19\t(0.24s)\n",
      "query 15 :\tr : 0.10\tp : 0.10\tf1 : 0.10\t(0.25s)\n",
      "query 16 :\tr : 0.12\tp : 0.20\tf1 : 0.15\t(0.24s)\n",
      "query 17 :\tr : 0.19\tp : 0.30\tf1 : 0.23\t(0.25s)\n",
      "query 18 :\tr : 0.18\tp : 0.20\tf1 : 0.19\t(0.25s)\n",
      "query 19 :\tr : 0.45\tp : 0.50\tf1 : 0.48\t(0.28s)\n",
      "query 20 :\tr : 0.00\tp : 0.00\tf1 : 0.00\t(0.27s)\n",
      "query 21 :\tr : 0.00\tp : 0.00\tf1 : 0.00\t(0.27s)\n",
      "query 22 :\tr : 0.53\tp : 0.90\tf1 : 0.67\t(0.26s)\n",
      "query 23 :\tr : 0.00\tp : 0.00\tf1 : 0.00\t(0.27s)\n",
      "query 24 :\tr : 0.15\tp : 0.20\tf1 : 0.17\t(0.24s)\n",
      "query 25 :\tr : 0.06\tp : 0.30\tf1 : 0.10\t(0.25s)\n",
      "query 26 :\tr : 0.23\tp : 0.70\tf1 : 0.35\t(0.24s)\n",
      "query 27 :\tr : 0.28\tp : 0.80\tf1 : 0.41\t(0.27s)\n",
      "query 28 :\tr : 0.80\tp : 0.40\tf1 : 0.53\t(0.24s)\n",
      "query 29 :\tr : 0.53\tp : 1.00\tf1 : 0.69\t(0.28s)\n",
      "query 30 :\tr : 0.25\tp : 0.10\tf1 : 0.14\t(0.28s)\n",
      "query 31 :\tr : 1.00\tp : 0.20\tf1 : 0.33\t(0.29s)\n",
      "query 32 :\tr : 0.67\tp : 0.20\tf1 : 0.31\t(0.28s)\n",
      "query 33 :\tr : 0.00\tp : 0.00\tf1 : 0.00\t(0.30s)\n",
      "query 34 :\tr : NaN\t\tp : 0.00\tf1 : 0.00\t(0.28s)\n",
      "query 35 :\tr : NaN\t\tp : 0.00\tf1 : 0.00\t(0.27s)\n",
      "query 36 :\tr : 0.15\tp : 0.30\tf1 : 0.20\t(0.26s)\n",
      "query 37 :\tr : 0.17\tp : 0.20\tf1 : 0.18\t(0.27s)\n",
      "query 38 :\tr : 0.38\tp : 0.60\tf1 : 0.46\t(0.27s)\n",
      "query 39 :\tr : 0.25\tp : 0.30\tf1 : 0.27\t(0.26s)\n",
      "query 40 :\tr : 0.40\tp : 0.40\tf1 : 0.40\t(0.25s)\n",
      "query 41 :\tr : NaN\t\tp : 0.00\tf1 : 0.00\t(0.28s)\n",
      "query 42 :\tr : 0.00\tp : 0.00\tf1 : 0.00\t(0.24s)\n",
      "query 43 :\tr : 0.07\tp : 0.30\tf1 : 0.12\t(0.24s)\n",
      "query 44 :\tr : 0.12\tp : 0.20\tf1 : 0.15\t(0.28s)\n",
      "query 45 :\tr : 0.27\tp : 0.70\tf1 : 0.39\t(0.25s)\n",
      "query 46 :\tr : NaN\t\tp : 0.00\tf1 : 0.00\t(0.26s)\n",
      "query 47 :\tr : NaN\t\tp : 0.00\tf1 : 0.00\t(0.27s)\n",
      "query 48 :\tr : 0.08\tp : 0.10\tf1 : 0.09\t(0.28s)\n",
      "query 49 :\tr : 0.25\tp : 0.20\tf1 : 0.22\t(0.28s)\n",
      "query 50 :\tr : NaN\t\tp : 0.00\tf1 : 0.00\t(0.27s)\n",
      "query 51 :\tr : NaN\t\tp : 0.00\tf1 : 0.00\t(0.28s)\n",
      "query 52 :\tr : NaN\t\tp : 0.00\tf1 : 0.00\t(0.27s)\n",
      "query 53 :\tr : NaN\t\tp : 0.00\tf1 : 0.00\t(0.25s)\n",
      "query 54 :\tr : NaN\t\tp : 0.00\tf1 : 0.00\t(0.27s)\n",
      "query 55 :\tr : NaN\t\tp : 0.00\tf1 : 0.00\t(0.24s)\n",
      "query 56 :\tr : NaN\t\tp : 0.00\tf1 : 0.00\t(0.26s)\n",
      "query 57 :\tr : 1.00\tp : 0.10\tf1 : 0.18\t(0.26s)\n",
      "query 58 :\tr : 0.23\tp : 0.70\tf1 : 0.35\t(0.25s)\n",
      "query 59 :\tr : 0.16\tp : 0.70\tf1 : 0.26\t(0.25s)\n",
      "query 60 :\tr : 0.19\tp : 0.50\tf1 : 0.27\t(0.25s)\n",
      "query 61 :\tr : 0.16\tp : 0.50\tf1 : 0.24\t(0.25s)\n",
      "query 62 :\tr : 0.00\tp : 0.00\tf1 : 0.00\t(0.24s)\n",
      "query 63 :\tr : 0.33\tp : 0.40\tf1 : 0.36\t(0.26s)\n",
      "query 64 :\tr : 0.00\tp : 0.00\tf1 : 0.00\t(0.24s)\n"
     ]
    }
   ],
   "source": [
    "query_truth, query_texts = read_truth(path_to_qrels, query_path=path_to_query)\n",
    "\n",
    "result_queries = []\n",
    "for id_query in range(1, len(query_texts)):\n",
    "    result = short_query_report(vec_index, id_query, query_texts[id_query], truth_queries[id_query])\n",
    "    result_queries += [result]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Index booléen sur CS276\n",
    "L'index booléen pour CS276 marche exactement de la même manière que pour CACM si ce n'est qu'il prend plusieurs heures à être généré. Les"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Loading index matrices\n",
      "Loading dictionaries\n",
      "Index built in 2.1s\n",
      "Warning : word \"setl\" not in the index\n",
      "['__UNK__', 'or', 'high', 'and', 'level', 'and', 'languag']\n",
      "or\n",
      "\t__UNK__\n",
      "\tand\n",
      "\t\thigh\n",
      "\t\tand\n",
      "\t\t\tlevel\n",
      "\t\t\tlanguag\n",
      "Query treated in 0.1s\n",
      "Docs found for query : SETL or (high and level and language)\n",
      "[ 1467  1991  2935  3399  3979  4402  5604  5663  6368  6484  6869  9566\n",
      " 10582 10723 10885 10913 10981 11201 11329 11412 12072 13191 13306 13582\n",
      " 13872 14504 15479 15578 15927 16194 16319 16375 16600 16613 16773 16998\n",
      " 17418 17520 17723 17896 18644 18718 19146 21099 21856 21889 22243 22275\n",
      " 22632 23037 23118 23625 24306 25666 26072 26110 26961 27898 28001 28154\n",
      " 28962 29441 29780 30539 30716 30791 30956 31496 31593 31659 31717 31804\n",
      " 31876 31914 32007 32078 32662 32958 32965 33023 33114 33971 34100 34174\n",
      " 34264 34515 34563 34650 35752 36004 36025 36079 37046 37302 37499 37620\n",
      " 37861 37894 37979 38167 38228 38391 38867 38999 39106 39133 39241 39571\n",
      " 39825 40369 40444 40936 41988 42509 43606 44493 45509 47174 47314 47673\n",
      " 47874 47958 48041 48596 49667 50850 51864 53031 53849 54591 57361 58862\n",
      " 59872 61699 62498 62718 62744 63024 63211 64540 64842 65008 65632 65688\n",
      " 66914 67782 68108 68491 73009 75803 76399 77887 80222 80673 80700 81254\n",
      " 81531 81933 81987 84036 84519 85619 86471 89914 90758 90935 91330 92100\n",
      " 92946 94734 94737 94975 95302 96195 96276 97431 97909 97994]\n"
     ]
    }
   ],
   "source": [
    "index = BooleanIndex()\n",
    "start = time()\n",
    "# index.build_cs276(directory_name=path_to_cs276_dir)\n",
    "index.load_cs276_index()\n",
    "print(\"Index built in {:.1f}s\".format(time()-start))\n",
    "start = time()\n",
    "\n",
    "query = \"SETL or (high and level and language)\"\n",
    "docs = index.treat_query(query, show_tree=True)\n",
    "print(\"Query treated in {:.1f}s\".format(time()-start))\n",
    "print(\"Docs found for query : {}\".format(query))\n",
    "print(docs)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## CS276 : Index vectoriel"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "VectorizedIndex = BooleanIndex()\n",
    "start = time()\n",
    "index.build_cs276(path_to_cs276_dir)\n",
    "print(\"Index built in {:.1f}s\".format(time()-start))\n"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.6.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}