examples/providers/pinecone/semantic-search.ipynb

{
 "cells": [
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Using the Pinecone Retrieval App\n",
    "\n",
    "In this walkthrough we will see how to use the retrieval API with a Pinecone datastore for *semantic search / question-answering*.\n",
    "\n",
    "Before running this notebook you should have already initialized the retrieval API and have it running locally or elsewhere. The full instructions for doing this are found in the [project README]().\n",
    "\n",
    "We will summarize the instructions (specific to the Pinecone datastore) before moving on to the walkthrough."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## App Quickstart"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "1. Install Python 3.10 if not already installed.\n",
    "\n",
    "2. Clone the `retrieval-app` repository:\n",
    "\n",
    "```\n",
    "git clone git@github.com:openai/retrieval-app.git\n",
    "```\n",
    "\n",
    "3. Navigate to the app directory:\n",
    "\n",
    "```\n",
    "cd /path/to/retrieval-app\n",
    "```\n",
    "\n",
    "4. Install `poetry`:\n",
    "\n",
    "```\n",
    "pip install poetry\n",
    "```\n",
    "\n",
    "5. Create a new virtual environment:\n",
    "\n",
    "```\n",
    "poetry env use python3.10\n",
    "```\n",
    "\n",
    "6. Install the `retrieval-app` dependencies:\n",
    "\n",
    "```\n",
    "poetry install\n",
    "```\n",
    "\n",
    "7. Set app environment variables:\n",
    "\n",
    "* `BEARER_TOKEN`: Secret token used by the app to authorize incoming requests. We will later include this in the request `headers`. The token can be generated however you prefer, such as using [jwt.io](https://jwt.io/).\n",
    "\n",
    "* `OPENAI_API_KEY`: The OpenAI API key used for generating embeddings with the OpenAI embeddings model. [Get an API key here](https://platform.openai.com/account/api-keys)!\n",
    "\n",
    "8. Set Pinecone-specific environment variables:\n",
    "\n",
    "* `DATASTORE`: set to `pinecone`.\n",
    "\n",
    "* `PINECONE_API_KEY`: Set to your Pinecone API key. This requires a free Pinecone account and can be [found in the Pinecone console](https://app.pinecone.io/).\n",
    "\n",
    "* `PINECONE_ENVIRONMENT`: Set to your Pinecone environment, looks like `us-east1-gcp`, `us-west1-aws`, and can be found next to your API key in the [Pinecone console](https://app.pinecone.io/).\n",
    "\n",
    "* `PINECONE_INDEX`: Set this to your chosen index name. The name you choose is your choice, we just recommend setting it to something descriptive like `\"openai-retrieval-app\"`. *Note that index names are restricted to alphanumeric characters, `\"-\"`, and can contain a maximum of 45 characters.*\n",
    "\n",
    "8. Run the app with:\n",
    "\n",
    "```\n",
    "poetry run start\n",
    "```\n",
    "\n",
    "If running the app locally you should see something like:\n",
    "\n",
    "```\n",
    "INFO:     Uvicorn running on http://0.0.0.0:8000\n",
    "INFO:     Application startup complete.\n",
    "```\n",
    "\n",
    "In that case, the app is automatically connected to our index (specified by `PINECONE_INDEX`), if no index with that name existed beforehand, the app creates one for us.\n",
    "\n",
    "Now we're ready to move on to populating our index with some data."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Required Libraries"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "There are a few Python libraries we must `pip install` for this notebook to run, those are:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!pip install -qU datasets pandas tqdm"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparing Data"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In this example, we will use the **S**tanford **Qu**estion **A**nswering **D**ataset (SQuAD), which we download from Hugging Face Datasets."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "Found cached dataset squad (/Users/jamesbriggs/.cache/huggingface/datasets/squad/plain_text/1.0.0/d6ec3ceb99ca480ce37cdd35555d6cb2511d223b9150cce08a837ef62ffea453)\n"
     ]
    },
    {
     "data": {
      "text/plain": [
       "Dataset({\n",
       "    features: ['id', 'title', 'context', 'question', 'answers'],\n",
       "    num_rows: 87599\n",
       "})"
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from datasets import load_dataset\n",
    "\n",
    "data = load_dataset(\"squad\", split=\"train\")\n",
    "data"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Convert to Pandas dataframe for easier preprocessing steps."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>context</th>\n",
       "      <th>question</th>\n",
       "      <th>answers</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5733be284776f41900661182</td>\n",
       "      <td>University_of_Notre_Dame</td>\n",
       "      <td>Architecturally, the school has a Catholic cha...</td>\n",
       "      <td>To whom did the Virgin Mary allegedly appear i...</td>\n",
       "      <td>{'text': ['Saint Bernadette Soubirous'], 'answ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>5733be284776f4190066117f</td>\n",
       "      <td>University_of_Notre_Dame</td>\n",
       "      <td>Architecturally, the school has a Catholic cha...</td>\n",
       "      <td>What is in front of the Notre Dame Main Building?</td>\n",
       "      <td>{'text': ['a copper statue of Christ'], 'answe...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>5733be284776f41900661180</td>\n",
       "      <td>University_of_Notre_Dame</td>\n",
       "      <td>Architecturally, the school has a Catholic cha...</td>\n",
       "      <td>The Basilica of the Sacred heart at Notre Dame...</td>\n",
       "      <td>{'text': ['the Main Building'], 'answer_start'...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>5733be284776f41900661181</td>\n",
       "      <td>University_of_Notre_Dame</td>\n",
       "      <td>Architecturally, the school has a Catholic cha...</td>\n",
       "      <td>What is the Grotto at Notre Dame?</td>\n",
       "      <td>{'text': ['a Marian place of prayer and reflec...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>5733be284776f4190066117e</td>\n",
       "      <td>University_of_Notre_Dame</td>\n",
       "      <td>Architecturally, the school has a Catholic cha...</td>\n",
       "      <td>What sits on top of the Main Building at Notre...</td>\n",
       "      <td>{'text': ['a golden statue of the Virgin Mary'...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                         id                     title  \\\n",
       "0  5733be284776f41900661182  University_of_Notre_Dame   \n",
       "1  5733be284776f4190066117f  University_of_Notre_Dame   \n",
       "2  5733be284776f41900661180  University_of_Notre_Dame   \n",
       "3  5733be284776f41900661181  University_of_Notre_Dame   \n",
       "4  5733be284776f4190066117e  University_of_Notre_Dame   \n",
       "\n",
       "                                             context  \\\n",
       "0  Architecturally, the school has a Catholic cha...   \n",
       "1  Architecturally, the school has a Catholic cha...   \n",
       "2  Architecturally, the school has a Catholic cha...   \n",
       "3  Architecturally, the school has a Catholic cha...   \n",
       "4  Architecturally, the school has a Catholic cha...   \n",
       "\n",
       "                                            question  \\\n",
       "0  To whom did the Virgin Mary allegedly appear i...   \n",
       "1  What is in front of the Notre Dame Main Building?   \n",
       "2  The Basilica of the Sacred heart at Notre Dame...   \n",
       "3                  What is the Grotto at Notre Dame?   \n",
       "4  What sits on top of the Main Building at Notre...   \n",
       "\n",
       "                                             answers  \n",
       "0  {'text': ['Saint Bernadette Soubirous'], 'answ...  \n",
       "1  {'text': ['a copper statue of Christ'], 'answe...  \n",
       "2  {'text': ['the Main Building'], 'answer_start'...  \n",
       "3  {'text': ['a Marian place of prayer and reflec...  \n",
       "4  {'text': ['a golden statue of the Virgin Mary'...  "
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = data.to_pandas()\n",
    "data.head()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The dataset contains a lot of duplicate `context` paragraphs, this is because each `context` can have many relevant questions. We don't want these duplicates so we remove like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "18891\n"
     ]
    },
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>title</th>\n",
       "      <th>context</th>\n",
       "      <th>question</th>\n",
       "      <th>answers</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>5733be284776f41900661182</td>\n",
       "      <td>University_of_Notre_Dame</td>\n",
       "      <td>Architecturally, the school has a Catholic cha...</td>\n",
       "      <td>To whom did the Virgin Mary allegedly appear i...</td>\n",
       "      <td>{'text': ['Saint Bernadette Soubirous'], 'answ...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>5</th>\n",
       "      <td>5733bf84d058e614000b61be</td>\n",
       "      <td>University_of_Notre_Dame</td>\n",
       "      <td>As at most other universities, Notre Dame's st...</td>\n",
       "      <td>When did the Scholastic Magazine of Notre dame...</td>\n",
       "      <td>{'text': ['September 1876'], 'answer_start': [...</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>10</th>\n",
       "      <td>5733bed24776f41900661188</td>\n",
       "      <td>University_of_Notre_Dame</td>\n",
       "      <td>The university is the major seat of the Congre...</td>\n",
       "      <td>Where is the headquarters of the Congregation ...</td>\n",
       "      <td>{'text': ['Rome'], 'answer_start': [119]}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>15</th>\n",
       "      <td>5733a6424776f41900660f51</td>\n",
       "      <td>University_of_Notre_Dame</td>\n",
       "      <td>The College of Engineering was established in ...</td>\n",
       "      <td>How many BS level degrees are offered in the C...</td>\n",
       "      <td>{'text': ['eight'], 'answer_start': [487]}</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>20</th>\n",
       "      <td>5733a70c4776f41900660f64</td>\n",
       "      <td>University_of_Notre_Dame</td>\n",
       "      <td>All of Notre Dame's undergraduate students are...</td>\n",
       "      <td>What entity provides help with the management ...</td>\n",
       "      <td>{'text': ['Learning Resource Center'], 'answer...</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                          id                     title  \\\n",
       "0   5733be284776f41900661182  University_of_Notre_Dame   \n",
       "5   5733bf84d058e614000b61be  University_of_Notre_Dame   \n",
       "10  5733bed24776f41900661188  University_of_Notre_Dame   \n",
       "15  5733a6424776f41900660f51  University_of_Notre_Dame   \n",
       "20  5733a70c4776f41900660f64  University_of_Notre_Dame   \n",
       "\n",
       "                                              context  \\\n",
       "0   Architecturally, the school has a Catholic cha...   \n",
       "5   As at most other universities, Notre Dame's st...   \n",
       "10  The university is the major seat of the Congre...   \n",
       "15  The College of Engineering was established in ...   \n",
       "20  All of Notre Dame's undergraduate students are...   \n",
       "\n",
       "                                             question  \\\n",
       "0   To whom did the Virgin Mary allegedly appear i...   \n",
       "5   When did the Scholastic Magazine of Notre dame...   \n",
       "10  Where is the headquarters of the Congregation ...   \n",
       "15  How many BS level degrees are offered in the C...   \n",
       "20  What entity provides help with the management ...   \n",
       "\n",
       "                                              answers  \n",
       "0   {'text': ['Saint Bernadette Soubirous'], 'answ...  \n",
       "5   {'text': ['September 1876'], 'answer_start': [...  \n",
       "10          {'text': ['Rome'], 'answer_start': [119]}  \n",
       "15         {'text': ['eight'], 'answer_start': [487]}  \n",
       "20  {'text': ['Learning Resource Center'], 'answer...  "
      ]
     },
     "execution_count": 3,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = data.drop_duplicates(subset=[\"context\"])\n",
    "print(len(data))\n",
    "data.head()"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The format required by the apps `upsert` function is a list of documents like:\n",
    "\n",
    "```json\n",
    "[\n",
    "    {\n",
    "        \"id\": \"abc\",\n",
    "        \"text\": \"some important document text\",\n",
    "        \"metadata\": {\n",
    "            \"field1\": \"optional metadata goes here\",\n",
    "            \"field2\": 54\n",
    "        }\n",
    "    },\n",
    "    {\n",
    "        \"id\": \"123\",\n",
    "        \"text\": \"some other important text\",\n",
    "        \"metadata\": {\n",
    "            \"field1\": \"another metadata\",\n",
    "            \"field2\": 71,\n",
    "            \"field3\": \"not all metadatas need the same structure\"\n",
    "        }\n",
    "    }\n",
    "    ...\n",
    "]\n",
    "```\n",
    "\n",
    "Every document *must* have a `\"text\"` field. The `\"id\"` and `\"metadata\"` fields are optional.\n",
    "\n",
    "To create this format for our SQuAD data we do:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'id': '5733be284776f41900661182',\n",
       "  'text': 'Architecturally, the school has a Catholic character. Atop the Main Building\\'s gold dome is a golden statue of the Virgin Mary. Immediately in front of the Main Building and facing it, is a copper statue of Christ with arms upraised with the legend \"Venite Ad Me Omnes\". Next to the Main Building is the Basilica of the Sacred Heart. Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.',\n",
       "  'metadata': {'title': 'University_of_Notre_Dame'}},\n",
       " {'id': '5733bf84d058e614000b61be',\n",
       "  'text': \"As at most other universities, Notre Dame's students run a number of news media outlets. The nine student-run outlets include three newspapers, both a radio and television station, and several magazines and journals. Begun as a one-page journal in September 1876, the Scholastic magazine is issued twice monthly and claims to be the oldest continuous collegiate publication in the United States. The other magazine, The Juggler, is released twice a year and focuses on student literature and artwork. The Dome yearbook is published annually. The newspapers have varying publication interests, with The Observer published daily and mainly reporting university and other news, and staffed by students from both Notre Dame and Saint Mary's College. Unlike Scholastic and The Dome, The Observer is an independent publication and does not have a faculty advisor or any editorial oversight from the University. In 1987, when some students believed that The Observer began to show a conservative bias, a liberal newspaper, Common Sense was published. Likewise, in 2003, when other students believed that the paper showed a liberal bias, the conservative paper Irish Rover went into production. Neither paper is published as often as The Observer; however, all three are distributed to all students. Finally, in Spring 2008 an undergraduate journal for political science research, Beyond Politics, made its debut.\",\n",
       "  'metadata': {'title': 'University_of_Notre_Dame'}},\n",
       " {'id': '5733bed24776f41900661188',\n",
       "  'text': 'The university is the major seat of the Congregation of Holy Cross (albeit not its official headquarters, which are in Rome). Its main seminary, Moreau Seminary, is located on the campus across St. Joseph lake from the Main Building. Old College, the oldest building on campus and located near the shore of St. Mary lake, houses undergraduate seminarians. Retired priests and brothers reside in Fatima House (a former retreat center), Holy Cross House, as well as Columba Hall near the Grotto. The university through the Moreau Seminary has ties to theologian Frederick Buechner. While not Catholic, Buechner has praised writers from Notre Dame and Moreau Seminary created a Buechner Prize for Preaching.',\n",
       "  'metadata': {'title': 'University_of_Notre_Dame'}}]"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "documents = [\n",
    "    {\n",
    "        'id': r['id'],\n",
    "        'text': r['context'],\n",
    "        'metadata': {\n",
    "            'title': r['title']\n",
    "        }\n",
    "    } for r in data.to_dict(orient='records')\n",
    "]\n",
    "documents[:3]"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Indexing the Docs"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We're now ready to begin indexing (or *upserting*) our `documents`. To make these requests to the retrieval app API, we will need to provide authorization in the form of the `BEARER_TOKEN` we set earlier. We do this below:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "\n",
    "BEARER_TOKEN = os.environ.get(\"BEARER_TOKEN\") or \"BEARER_TOKEN_HERE\""
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Use the `BEARER_TOKEN` to create our authorization `headers`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [],
   "source": [
    "headers = {\n",
    "    \"Authorization\": f\"Bearer {BEARER_TOKEN}\"\n",
    "}"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We'll perform the upsert in batches of `batch_size`. Make sure that the `endpoint_url` variable is set to the correct location for your running *retrieval-app* API."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "8694da67455d4bb78cc778e49f69a872",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/10 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "from tqdm.auto import tqdm\n",
    "import requests\n",
    "from requests.adapters import HTTPAdapter, Retry\n",
    "\n",
    "batch_size = 100\n",
    "endpoint_url = \"http://localhost:8000\"\n",
    "s = requests.Session()\n",
    "\n",
    "# we setup a retry strategy to retry on 5xx errors\n",
    "retries = Retry(\n",
    "    total=5,  # number of retries before raising error\n",
    "    backoff_factor=0.1,\n",
    "    status_forcelist=[500, 502, 503, 504]\n",
    ")\n",
    "s.mount('http://', HTTPAdapter(max_retries=retries))\n",
    "\n",
    "for i in tqdm(range(0, len(documents), batch_size)):\n",
    "    i_end = min(len(documents), i+batch_size)\n",
    "    # make post request that allows up to 5 retries\n",
    "    res = s.post(\n",
    "        f\"{endpoint_url}/upsert\",\n",
    "        headers=headers,\n",
    "        json={\n",
    "            \"documents\": documents[i:i_end]\n",
    "        }\n",
    "    )"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "With that our SQuAD records have all been indexed and we can move on to querying."
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Making Queries"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To query the datastore all we need to do is pass one or more queries to the `/query` endpoint. We can take a few questions from SQuAD:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "18891"
      ]
     },
     "execution_count": 8,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "queries = data['question'].tolist()\n",
    "# format into the structure needed by the /query endpoint\n",
    "queries = [{'query': queries[i]} for i in range(len(queries))]\n",
    "len(queries)"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We will use just the first *three* questions:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'query': 'To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?'},\n",
       " {'query': 'When did the Scholastic Magazine of Notre dame begin publishing?'},\n",
       " {'query': 'Where is the headquarters of the Congregation of the Holy Cross?'}]"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "queries[:3]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<Response [200]>"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "res = requests.post(\n",
    "    \"http://0.0.0.0:8000/query\",\n",
    "    headers=headers,\n",
    "    json={\n",
    "        'queries': queries[:3]\n",
    "    }\n",
    ")\n",
    "res"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can loop through the responses and see the results returned for each query:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "----------------------------------------------------------------------\n",
      "To whom did the Virgin Mary allegedly appear in 1858 in Lourdes France?\n",
      "\n",
      "0.83: Architecturally, the school has a Catholic character. Atop the Main Building's gold dome is a golden statue of the Virgin Mary. Immediately in front of the Main Building and facing it, is a copper statue of Christ with arms upraised with the legend \"Venite Ad Me Omnes\". Next to the Main Building is the Basilica of the Sacred Heart. Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.\n",
      "0.81: Within the white inescutcheon, the five quinas (small blue shields) with their five white bezants representing the five wounds of Christ (Portuguese: Cinco Chagas) when crucified and are popularly associated with the \"Miracle of Ourique\". The story associated with this miracle tells that before the Battle of Ourique (25 July 1139), an old hermit appeared before Count Afonso Henriques (future Afonso I) as a divine messenger. He foretold Afonso's victory and assured him that God was watching over him and his peers. The messenger advised him to walk away from his camp, alone, if he heard a nearby chapel bell tolling, in the following night. In doing so, he witnessed an apparition of Jesus on the cross. Ecstatic, Afonso heard Jesus promising victories for the coming battles, as well as God's wish to act through Afonso, and his descendants, in order to create an empire which would carry His name to unknown lands, thus choosing the Portuguese to perform great tasks.\n",
      "0.79: In 1842, the Bishop of Vincennes, Célestine Guynemer de la Hailandière, offered land to Father Edward Sorin of the Congregation of the Holy Cross, on the condition that he build a college in two years. Fr. Sorin arrived on the site with eight Holy Cross brothers from France and Ireland on November 26, 1842, and began the school using Father Stephen Badin's old log chapel. He soon erected additional buildings, including Old College, the first church, and the first main building. They immediately acquired two students and set about building additions to the campus.\n",
      "0.79: Because of its Catholic identity, a number of religious buildings stand on campus. The Old College building has become one of two seminaries on campus run by the Congregation of Holy Cross. The current Basilica of the Sacred Heart is located on the spot of Fr. Sorin's original church, which became too small for the growing college. It is built in French Revival style and it is decorated by stained glass windows imported directly from France. The interior was painted by Luigi Gregori, an Italian painter invited by Fr. Sorin to be artist in residence. The Basilica also features a bell tower with a carillon. Inside the church there are also sculptures by Ivan Mestrovic. The Grotto of Our Lady of Lourdes, which was built in 1896, is a replica of the original in Lourdes, France. It is very popular among students and alumni as a place of prayer and meditation, and it is considered one of the most beloved spots on campus.\n",
      "0.78: The funeral, held at the Church of the Madeleine in Paris, was delayed almost two weeks, until 30 October. Entrance was restricted to ticket holders as many people were expected to attend. Over 3,000 people arrived without invitations, from as far as London, Berlin and Vienna, and were excluded.\n",
      "----------------------------------------------------------------------\n",
      "\n",
      "\n",
      "----------------------------------------------------------------------\n",
      "When did the Scholastic Magazine of Notre dame begin publishing?\n",
      "\n",
      "0.88: As at most other universities, Notre Dame's students run a number of news media outlets. The nine student-run outlets include three newspapers, both a radio and television station, and several magazines and journals. Begun as a one-page journal in September 1876, the Scholastic magazine is issued twice monthly and claims to be the oldest continuous collegiate publication in the United States. The other magazine, The Juggler, is released twice a year and focuses on student literature and artwork. The Dome yearbook is published annually. The newspapers have varying publication interests, with The Observer published daily and mainly reporting university and other news, and staffed by students from both Notre Dame and Saint Mary's College. Unlike Scholastic and The Dome, The Observer is an independent publication and does not have a faculty advisor or any editorial oversight from the University. In 1987, when some students believed that The Observer began to show a conservative bias, a liberal newspaper, Common Sense was published. Likewise, in 2003, when other students believed that the paper showed a liberal bias, the conservative paper Irish Rover went into production. Neither paper is published as often as The Observer; however, all three are distributed to all students.\n",
      "0.83: In 1919 Father James Burns became president of Notre Dame, and in three years he produced an academic revolution that brought the school up to national standards by adopting the elective system and moving away from the university's traditional scholastic and classical emphasis. By contrast, the Jesuit colleges, bastions of academic conservatism, were reluctant to move to a system of electives. Their graduates were shut out of Harvard Law School for that reason. Notre Dame continued to grow over the years, adding more colleges, programs, and sports teams. By 1921, with the addition of the College of Commerce, Notre Dame had grown from a small college to a university with five colleges and a professional law school. The university continued to expand and add new residence halls and buildings with each subsequent president.\n",
      "0.83: The rise of Hitler and other dictators in the 1930s forced numerous Catholic intellectuals to flee Europe; president John O'Hara brought many to Notre Dame. From Germany came Anton-Hermann Chroust (1907–1982) in classics and law, and Waldemar Gurian a German Catholic intellectual of Jewish descent. Positivism dominated American intellectual life in the 1920s onward but in marked contrast, Gurian received a German Catholic education and wrote his doctoral dissertation under Max Scheler. Ivan Meštrović (1883–1962), a renowned sculptor, brought Croatian culture to campus, 1955–62. Yves Simon (1903–61), brought to ND in the 1940s the insights of French studies in the Aristotelian-Thomistic tradition of philosophy; his own teacher Jacques Maritain (1882–73) was a frequent visitor to campus.\n",
      "0.82: In the 18 years under the presidency of Edward Malloy, C.S.C., (1987–2005), there was a rapid growth in the school's reputation, faculty, and resources. He increased the faculty by more than 500 professors; the academic quality of the student body has improved dramatically, with the average SAT score rising from 1240 to 1360; the number of minority students more than doubled; the endowment grew from $350 million to more than $3 billion; the annual operating budget rose from $177 million to more than $650 million; and annual research funding improved from $15 million to more than $70 million. Notre Dame's most recent[when?] capital campaign raised $1.1 billion, far exceeding its goal of $767 million, and is the largest in the history of Catholic higher education.\n",
      "0.82: The Rev. John J. Cavanaugh, C.S.C. served as president from 1946 to 1952. Cavanaugh's legacy at Notre Dame in the post-war years was devoted to raising academic standards and reshaping the university administration to suit it to an enlarged educational mission and an expanded student body and stressing advanced studies and research at a time when Notre Dame quadrupled in student census, undergraduate enrollment increased by more than half, and graduate student enrollment grew fivefold. Cavanaugh also established the Lobund Institute for Animal Studies and Notre Dame's Medieval Institute. Cavanaugh also presided over the construction of the Nieuwland Science Hall, Fisher Hall, and the Morris Inn, as well as the Hall of Liberal Arts (now O'Shaughnessy Hall), made possible by a donation from I.A. O'Shaughnessy, at the time the largest ever made to an American Catholic university.\n",
      "----------------------------------------------------------------------\n",
      "\n",
      "\n",
      "----------------------------------------------------------------------\n",
      "Where is the headquarters of the Congregation of the Holy Cross?\n",
      "\n",
      "0.88: The university is the major seat of the Congregation of Holy Cross (albeit not its official headquarters, which are in Rome). Its main seminary, Moreau Seminary, is located on the campus across St. Joseph lake from the Main Building. Old College, the oldest building on campus and located near the shore of St. Mary lake, houses undergraduate seminarians. Retired priests and brothers reside in Fatima House (a former retreat center), Holy Cross House, as well as Columba Hall near the Grotto. The university through the Moreau Seminary has ties to theologian Frederick Buechner. While not Catholic, Buechner has praised writers from Notre Dame and Moreau Seminary created a Buechner Prize for Preaching.\n",
      "0.84: In 1842, the Bishop of Vincennes, Célestine Guynemer de la Hailandière, offered land to Father Edward Sorin of the Congregation of the Holy Cross, on the condition that he build a college in two years. Fr. Sorin arrived on the site with eight Holy Cross brothers from France and Ireland on November 26, 1842, and began the school using Father Stephen Badin's old log chapel. He soon erected additional buildings, including Old College, the first church, and the first main building. They immediately acquired two students and set about building additions to the campus.\n",
      "0.84: Because of its Catholic identity, a number of religious buildings stand on campus. The Old College building has become one of two seminaries on campus run by the Congregation of Holy Cross. The current Basilica of the Sacred Heart is located on the spot of Fr. Sorin's original church, which became too small for the growing college. It is built in French Revival style and it is decorated by stained glass windows imported directly from France. The interior was painted by Luigi Gregori, an Italian painter invited by Fr. Sorin to be artist in residence. The Basilica also features a bell tower with a carillon. Inside the church there are also sculptures by Ivan Mestrovic. The Grotto of Our Lady of Lourdes, which was built in 1896, is a replica of the original in Lourdes, France. It is very popular among students and alumni as a place of prayer and meditation, and it is considered one of the most beloved spots on campus.\n",
      "0.84: The university is affiliated with the Congregation of Holy Cross (Latin: Congregatio a Sancta Cruce, abbreviated postnominals: \"CSC\"). While religious affiliation is not a criterion for admission, more than 93% of students identify as Christian, with over 80% of the total being Catholic. Collectively, Catholic Mass is celebrated over 100 times per week on campus, and a large campus ministry program provides for the faith needs of the community. There are multitudes of religious statues and artwork around campus, most prominent of which are the statue of Mary on the Main Building, the Notre Dame Grotto, and the Word of Life mural on Hesburgh Library depicting Christ as a teacher. Additionally, every classroom displays a crucifix. There are many religious clubs (catholic and non-Catholic) at the school, including Council #1477 of the Knights of Columbus (KOC), Baptist Collegiate Ministry (BCM), Jewish Club, Muslim Student Association, Orthodox Christian Fellowship, The Mormon Club, and many more. The Notre Dame KofC are known for being the first collegiate council of KofC, operating a charitable concession stand during every home football game and owning their own building on campus which can be used as a cigar lounge.\n",
      "0.83: Architecturally, the school has a Catholic character. Atop the Main Building's gold dome is a golden statue of the Virgin Mary. Immediately in front of the Main Building and facing it, is a copper statue of Christ with arms upraised with the legend \"Venite Ad Me Omnes\". Next to the Main Building is the Basilica of the Sacred Heart. Immediately behind the basilica is the Grotto, a Marian place of prayer and reflection. It is a replica of the grotto at Lourdes, France where the Virgin Mary reputedly appeared to Saint Bernadette Soubirous in 1858. At the end of the main drive (and in a direct line that connects through 3 statues and the Gold Dome), is a simple, modern stone statue of Mary.\n",
      "----------------------------------------------------------------------\n",
      "\n",
      "\n"
     ]
    }
   ],
   "source": [
    "for query_result in res.json()['results']:\n",
    "    query = query_result['query']\n",
    "    answers = []\n",
    "    scores = []\n",
    "    for result in query_result['results']:\n",
    "        answers.append(result['text'])\n",
    "        scores.append(round(result['score'], 2))\n",
    "    print(\"-\"*70+\"\\n\"+query+\"\\n\\n\"+\"\\n\".join([f\"{s}: {a}\" for a, s in zip(answers, scores)])+\"\\n\"+\"-\"*70+\"\\n\\n\")"
   ]
  },
  {
   "attachments": {},
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The top results are all relevant as we would have hoped. With that we've finished. The retrieval app API can be shut down, and to save resources the Pinecone index can be deleted within the [Pinecone console](https://app.pinecone.io/)."
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "chatgpt-retrieval-plugin-S7h-2AWq-py3.10",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.10.8"
  },
  "orig_nbformat": 4,
  "vscode": {
   "interpreter": {
    "hash": "1979a773a5778de9a5fa593a629dff0ab3c80c2563810d3e6a8dfb123dc01c7d"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}