intro_workshop_finished.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "c641751f-f3d4-47b6-806a-0e55c0b73405",
   "metadata": {},
   "source": [
    "## Weaviate (intro) workshop\n",
    "\n",
    "<a target=\"_blank\" href=\"https://colab.research.google.com/github/weaviate-tutorials/intro-workshop/blob/main/workshop.ipynb\">\n",
    "  <img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/>\n",
    "</a>\n",
    "\n",
    "### Goals:\n",
    "\n",
    "#### What you will see:\n",
    "\n",
    "\n",
    "- Create a vector database with Weaviate,\n",
    "- Add data to the database, and\n",
    "- Interact with the data, including searching, and using LLMs with your data in Weaviate\n",
    "\n",
    "### You will learn:\n",
    "\n",
    "- What Weaviate is,\n",
    "- How it stores the data (based on its \"meaning\"), and\n",
    "- What you can do with Weaviate, like semantic searches, and using LLMs to transform data."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5ba26bc3-0958-4051-b31e-9ce27555a8b7",
   "metadata": {},
   "source": [
    "## Preparation: Get the data\n",
    "\n",
    "We'll use a dataset of movies from TMDB. \n",
    "\n",
    "Pre-processed version: \"./data/movies.csv\"\n"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1633e880-94bb-4942-9fe1-70ffd2a7afd3",
   "metadata": {},
   "source": [
    "Load (or download) the data, and preview it"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "4680be3b-e00f-4c3c-9d42-d9bd8ef1e5a9",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-08-12T20:50:09.186617Z",
     "start_time": "2023-08-12T20:50:09.042514Z"
    },
    "executionCancelledAt": null,
    "executionTime": 1192,
    "lastExecutedAt": 1692032184432,
    "lastScheduledRunId": null,
    "lastSuccessfullyExecutedCode": "import requests\nimport json\n\n# Load the data locally\nwith open(\"jeopardy_1k.json\", \"r\") as f:\n    raw_data = f.read()\n\n# Or download it from GitHub\nresponse = requests.get('https://raw.githubusercontent.com/databyjp/wv_demo_uploader/main/weaviate_datasets/data/jeopardy_1k.json')\nraw_data = response.text\n\n# Parse the JSON and preview it\ndata = json.loads(raw_data)\nprint(type(data), len(data))\nprint(json.dumps(data[0], indent=2))",
    "outputsMetadata": {
     "0": {
      "height": 217,
      "type": "stream"
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>backdrop_path</th>\n",
       "      <th>genre_ids</th>\n",
       "      <th>id</th>\n",
       "      <th>original_language</th>\n",
       "      <th>original_title</th>\n",
       "      <th>overview</th>\n",
       "      <th>popularity</th>\n",
       "      <th>poster_path</th>\n",
       "      <th>release_date</th>\n",
       "      <th>title</th>\n",
       "      <th>video</th>\n",
       "      <th>vote_average</th>\n",
       "      <th>vote_count</th>\n",
       "      <th>year</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>/rH0DPF7pB35jxLxKb3JRUgCrrnp.jpg</td>\n",
       "      <td>[10751, 14, 16, 10749]</td>\n",
       "      <td>11224</td>\n",
       "      <td>en</td>\n",
       "      <td>Cinderella</td>\n",
       "      <td>Cinderella has faith her dreams of a better li...</td>\n",
       "      <td>100.819</td>\n",
       "      <td>/avz6S9HYWs4O8Oe4PenBFNX4uDi.jpg</td>\n",
       "      <td>1950-02-22</td>\n",
       "      <td>Cinderella</td>\n",
       "      <td>False</td>\n",
       "      <td>7.044</td>\n",
       "      <td>6523</td>\n",
       "      <td>1950</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>/p47ihFj4A7EpBjmPHdTj4ipyq1S.jpg</td>\n",
       "      <td>[18]</td>\n",
       "      <td>599</td>\n",
       "      <td>en</td>\n",
       "      <td>Sunset Boulevard</td>\n",
       "      <td>A hack screenwriter writes a screenplay for a ...</td>\n",
       "      <td>57.740</td>\n",
       "      <td>/sC4Dpmn87oz9AuxZ15Lmip0Ftgr.jpg</td>\n",
       "      <td>1950-08-10</td>\n",
       "      <td>Sunset Boulevard</td>\n",
       "      <td>False</td>\n",
       "      <td>8.312</td>\n",
       "      <td>2485</td>\n",
       "      <td>1950</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>/zyO6j74DKMWfp5snWg6Hwo0T3Mz.jpg</td>\n",
       "      <td>[80, 18, 9648]</td>\n",
       "      <td>548</td>\n",
       "      <td>ja</td>\n",
       "      <td>羅生門</td>\n",
       "      <td>Brimming with action while incisively examinin...</td>\n",
       "      <td>21.011</td>\n",
       "      <td>/vL7Xw04nFMHwnvXRFCmYYAzMUvY.jpg</td>\n",
       "      <td>1950-08-26</td>\n",
       "      <td>Rashomon</td>\n",
       "      <td>False</td>\n",
       "      <td>8.091</td>\n",
       "      <td>2121</td>\n",
       "      <td>1950</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>/b4yiLlIFuiULuuLTxT0Pt1QyT6J.jpg</td>\n",
       "      <td>[16, 10751, 14, 12]</td>\n",
       "      <td>12092</td>\n",
       "      <td>en</td>\n",
       "      <td>Alice in Wonderland</td>\n",
       "      <td>On a golden afternoon, young Alice follows a W...</td>\n",
       "      <td>75.465</td>\n",
       "      <td>/20cvfwfaFqNbe9Fc3VEHJuPRxmn.jpg</td>\n",
       "      <td>1951-07-28</td>\n",
       "      <td>Alice in Wonderland</td>\n",
       "      <td>False</td>\n",
       "      <td>7.200</td>\n",
       "      <td>5697</td>\n",
       "      <td>1951</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>/mxf8hJJkHTCqZP3m4o8E1TtwHHs.jpg</td>\n",
       "      <td>[35, 10749]</td>\n",
       "      <td>872</td>\n",
       "      <td>en</td>\n",
       "      <td>Singin' in the Rain</td>\n",
       "      <td>In 1927 Hollywood, a silent film production co...</td>\n",
       "      <td>31.407</td>\n",
       "      <td>/w03EiJVHP8Un77boQeE7hg9DVdU.jpg</td>\n",
       "      <td>1952-04-09</td>\n",
       "      <td>Singin' in the Rain</td>\n",
       "      <td>False</td>\n",
       "      <td>8.200</td>\n",
       "      <td>3036</td>\n",
       "      <td>1952</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                      backdrop_path               genre_ids     id  \\\n",
       "0  /rH0DPF7pB35jxLxKb3JRUgCrrnp.jpg  [10751, 14, 16, 10749]  11224   \n",
       "1  /p47ihFj4A7EpBjmPHdTj4ipyq1S.jpg                    [18]    599   \n",
       "2  /zyO6j74DKMWfp5snWg6Hwo0T3Mz.jpg          [80, 18, 9648]    548   \n",
       "3  /b4yiLlIFuiULuuLTxT0Pt1QyT6J.jpg     [16, 10751, 14, 12]  12092   \n",
       "4  /mxf8hJJkHTCqZP3m4o8E1TtwHHs.jpg             [35, 10749]    872   \n",
       "\n",
       "  original_language       original_title  \\\n",
       "0                en           Cinderella   \n",
       "1                en     Sunset Boulevard   \n",
       "2                ja                  羅生門   \n",
       "3                en  Alice in Wonderland   \n",
       "4                en  Singin' in the Rain   \n",
       "\n",
       "                                            overview  popularity  \\\n",
       "0  Cinderella has faith her dreams of a better li...     100.819   \n",
       "1  A hack screenwriter writes a screenplay for a ...      57.740   \n",
       "2  Brimming with action while incisively examinin...      21.011   \n",
       "3  On a golden afternoon, young Alice follows a W...      75.465   \n",
       "4  In 1927 Hollywood, a silent film production co...      31.407   \n",
       "\n",
       "                        poster_path release_date                title  video  \\\n",
       "0  /avz6S9HYWs4O8Oe4PenBFNX4uDi.jpg   1950-02-22           Cinderella  False   \n",
       "1  /sC4Dpmn87oz9AuxZ15Lmip0Ftgr.jpg   1950-08-10     Sunset Boulevard  False   \n",
       "2  /vL7Xw04nFMHwnvXRFCmYYAzMUvY.jpg   1950-08-26             Rashomon  False   \n",
       "3  /20cvfwfaFqNbe9Fc3VEHJuPRxmn.jpg   1951-07-28  Alice in Wonderland  False   \n",
       "4  /w03EiJVHP8Un77boQeE7hg9DVdU.jpg   1952-04-09  Singin' in the Rain  False   \n",
       "\n",
       "   vote_average  vote_count  year  \n",
       "0         7.044        6523  1950  \n",
       "1         8.312        2485  1950  \n",
       "2         8.091        2121  1950  \n",
       "3         7.200        5697  1951  \n",
       "4         8.200        3036  1952  "
      ]
     },
     "execution_count": 1,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import pandas as pd\n",
    "\n",
    "# movie_df = pd.read_csv(\"./data/movies.csv\")\n",
    "movie_df = pd.read_csv(\"https://raw.githubusercontent.com/weaviate-tutorials/intro-workshop/main/data/movies.csv\")\n",
    "movie_df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d27642a1-e738-441c-9008-ce8cb267f28b",
   "metadata": {},
   "source": [
    "## Step 1: Connect to Weaviate"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "97a88ba7-6e35-4378-ad30-869537966f01",
   "metadata": {},
   "source": [
    "You can also use a hosted instance on Weaviate Cloud, or install Weaviate anywhere using the open-source distribution."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0dbfea6e",
   "metadata": {},
   "source": [
    "If you are using Cohere, or OpenAI, uncomment and update the relevant lines in the following cell with your actual keys. If yor using Ollama, you do not need to do anything."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "dbebb69b",
   "metadata": {},
   "outputs": [],
   "source": [
    "headers={\n",
    "    # \"X-Cohere-Api-Key\": \"<your_cohere_apikey>\",  # Replace this with your actual key\n",
    "    # \"X-OpenAI-Api-Key\": \"<your_openai_apikey>\",  # Replace this with your actual key\n",
    "}"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ba40b5a3-d645-493b-9c93-e652c1057870",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-08-12T20:50:10.463278Z",
     "start_time": "2023-08-12T20:50:10.020181Z"
    },
    "executionCancelledAt": null,
    "executionTime": null,
    "lastExecutedAt": null,
    "lastScheduledRunId": null,
    "lastSuccessfullyExecutedCode": null,
    "outputsMetadata": {
     "0": {
      "height": 97,
      "type": "stream"
     },
     "1": {
      "height": 257,
      "type": "stream"
     }
    },
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "import weaviate\n",
    "\n",
    "# If you have got Weaviate running locally with Kubernetes or Docker:\n",
    "client = weaviate.connect_to_local(\n",
    "    port=8080,  # Or 80 for Kubernetes instances\n",
    "    headers=headers\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "388d970d-79a2-4591-a5f3-c6c17406df2a",
   "metadata": {},
   "source": [
    "Retrieve Weaviate instance information to check our configuration."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "b8a4226e-41fd-4ad7-9416-dcda4a443d58",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-08-12T20:50:11.726853Z",
     "start_time": "2023-08-12T20:50:11.722015Z"
    },
    "collapsed": true,
    "executionCancelledAt": null,
    "executionTime": 48,
    "jupyter": {
     "outputs_hidden": true
    },
    "lastExecutedAt": 1692032185634,
    "lastScheduledRunId": null,
    "lastSuccessfullyExecutedCode": "jprint(client.get_meta())",
    "outputsMetadata": {
     "0": {
      "height": 597,
      "type": "stream"
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "{'hostname': 'http://[::]:8080',\n",
       " 'modules': {'generative-anthropic': {'documentationHref': 'https://docs.anthropic.com/en/api/getting-started',\n",
       "   'name': 'Generative Search - Anthropic'},\n",
       "  'generative-anyscale': {'documentationHref': 'https://docs.anyscale.com/endpoints/overview',\n",
       "   'name': 'Generative Search - Anyscale'},\n",
       "  'generative-aws': {'documentationHref': 'https://docs.aws.amazon.com/bedrock/latest/APIReference/welcome.html',\n",
       "   'name': 'Generative Search - AWS'},\n",
       "  'generative-cohere': {'documentationHref': 'https://docs.cohere.com/reference/chat',\n",
       "   'name': 'Generative Search - Cohere'},\n",
       "  'generative-mistral': {'documentationHref': 'https://docs.mistral.ai/api/',\n",
       "   'name': 'Generative Search - Mistral'},\n",
       "  'generative-octoai': {'documentationHref': 'https://octo.ai/docs/text-gen-solution/getting-started',\n",
       "   'name': 'Generative Search - OctoAI'},\n",
       "  'generative-ollama': {'documentationHref': 'https://github.com/ollama/ollama/blob/main/docs/api.md#generate-a-completion',\n",
       "   'name': 'Generative Search - Ollama'},\n",
       "  'generative-openai': {'documentationHref': 'https://platform.openai.com/docs/api-reference/completions',\n",
       "   'name': 'Generative Search - OpenAI'},\n",
       "  'generative-palm': {'documentationHref': 'https://cloud.google.com/vertex-ai/docs/generative-ai/chat/test-chat-prompts',\n",
       "   'name': 'Generative Search - Google PaLM'},\n",
       "  'multi2vec-palm': {'documentationHref': 'https://cloud.google.com/vertex-ai/generative-ai/docs/embeddings/get-multimodal-embeddings',\n",
       "   'name': 'Google PaLM Multimodal Module'},\n",
       "  'reranker-cohere': {'documentationHref': 'https://txt.cohere.com/rerank/',\n",
       "   'name': 'Reranker - Cohere'},\n",
       "  'reranker-voyageai': {'documentationHref': 'https://docs.voyageai.com/reference/reranker-api',\n",
       "   'name': 'Reranker - VoyageAI'},\n",
       "  'text2vec-aws': {'documentationHref': 'https://docs.aws.amazon.com/bedrock/latest/userguide/titan-embedding-models.html',\n",
       "   'name': 'AWS Module'},\n",
       "  'text2vec-cohere': {'documentationHref': 'https://docs.cohere.ai/embedding-wiki/',\n",
       "   'name': 'Cohere Module'},\n",
       "  'text2vec-huggingface': {'documentationHref': 'https://huggingface.co/docs/api-inference/detailed_parameters#feature-extraction-task',\n",
       "   'name': 'Hugging Face Module'},\n",
       "  'text2vec-jinaai': {'documentationHref': 'https://jina.ai/embeddings/',\n",
       "   'name': 'JinaAI Module'},\n",
       "  'text2vec-octoai': {'documentationHref': 'https://octo.ai/docs/text-gen-solution/getting-started',\n",
       "   'name': 'OctoAI Module'},\n",
       "  'text2vec-ollama': {'documentationHref': 'https://github.com/ollama/ollama/blob/main/docs/api.md#generate-embeddings',\n",
       "   'name': 'Ollama Module'},\n",
       "  'text2vec-openai': {'documentationHref': 'https://platform.openai.com/docs/guides/embeddings/what-are-embeddings',\n",
       "   'name': 'OpenAI Module'},\n",
       "  'text2vec-palm': {'documentationHref': 'https://cloud.google.com/vertex-ai/docs/generative-ai/embeddings/get-text-embeddings',\n",
       "   'name': 'Google PaLM Module'},\n",
       "  'text2vec-voyageai': {'documentationHref': 'https://docs.voyageai.com/docs/embeddings',\n",
       "   'name': 'VoyageAI Module'}},\n",
       " 'version': '1.26.6'}"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "client.get_meta()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "68b9706e-e11e-43f8-ae13-3741c5376b25",
   "metadata": {},
   "source": [
    "## Step 2: Add data to Weaviate"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5f01b498-455f-4e82-bdb3-df7b95363ca4",
   "metadata": {},
   "source": [
    "### Add collection definition"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "310a7ecf-0f04-48a3-b622-dd3673e8302f",
   "metadata": {},
   "source": [
    "The equivalent of a SQL \"table\", is called a \"collection\" in Weaviate, like they are in NoSQL databases."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ba03c144-e2a3-48cc-97db-590fbac55ec0",
   "metadata": {},
   "source": [
    "In case I created a demo collection - let's delete it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 40,
   "id": "976fdc87-ee1f-43cd-8db7-57b1fda5d697",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-08-12T20:50:23.339817Z",
     "start_time": "2023-08-12T20:50:23.303039Z"
    },
    "executionCancelledAt": null,
    "executionTime": 47,
    "lastExecutedAt": 1692032185681,
    "lastScheduledRunId": null,
    "lastSuccessfullyExecutedCode": "if client.schema.exists(\"Question\"):\n    client.schema.delete_class(\"Question\")"
   },
   "outputs": [],
   "source": [
    "client.collections.delete(\"Movie\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9279cefa-ecaf-487c-a090-43a14383e9a1",
   "metadata": {},
   "source": [
    "And create a new collection definition here.\n",
    "We'll set up a collection called \"Movie\" with:\n",
    "- Two \"named vectors\" -> which will save different \"meanings\" of the data,\n",
    "- A \"generative\" module -> which will allow us to use LLMs with our data, and\n",
    "- Properties to save our movie data (which are like SQL columns).\n",
    "    - Just the title, overview, year and popularity for now."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "id": "5da0db4a-9f72-402c-9c23-49f1500ea995",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2023-08-12T20:50:24.018171Z",
     "start_time": "2023-08-12T20:50:23.993529Z"
    },
    "executionCancelledAt": null,
    "executionTime": null,
    "lastExecutedAt": null,
    "lastScheduledRunId": null,
    "lastSuccessfullyExecutedCode": null,
    "outputsMetadata": {
     "0": {
      "height": 77,
      "type": "stream"
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "<weaviate.collections.collection.sync.Collection at 0x127705a90>"
      ]
     },
     "execution_count": 41,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from weaviate.classes.config import Configure, DataType, Property\n",
    "\n",
    "client.collections.create(\n",
    "    name=\"Movie\",\n",
    "    properties=[\n",
    "        Property(\n",
    "            name=\"title\",\n",
    "            data_type=DataType.TEXT,\n",
    "        ),\n",
    "        Property(\n",
    "            name=\"overview\",\n",
    "            data_type=DataType.TEXT,\n",
    "        ),\n",
    "        Property(\n",
    "            name=\"popularity\",\n",
    "            data_type=DataType.NUMBER,\n",
    "        ),\n",
    "        Property(\n",
    "            name=\"year\",\n",
    "            data_type=DataType.INT,\n",
    "        ),\n",
    "    ],\n",
    "    # ========================================\n",
    "    # For those using Ollama:\n",
    "    # ========================================\n",
    "    vectorizer_config=[\n",
    "        Configure.NamedVectors.text2vec_ollama(\n",
    "            name=\"title\",\n",
    "            source_properties=[\"title\"],\n",
    "            api_endpoint=\"http://host.docker.internal:11434\",  # If using Docker, use this to contact your local Ollama instance\n",
    "            model=\"nomic-embed-text\",  # The model to use, e.g. \"snowflake-arctic-embed\"\n",
    "        ),\n",
    "        Configure.NamedVectors.text2vec_ollama(\n",
    "            name=\"all_text\",\n",
    "            source_properties=[\"title\", \"overview\"],\n",
    "            api_endpoint=\"http://host.docker.internal:11434\",  # If using Docker, use this to contact your local Ollama instance\n",
    "            model=\"nomic-embed-text\",  # The model to use, e.g. \"snowflake-arctic-embed\"\n",
    "        ),\n",
    "    ],\n",
    "    generative_config=Configure.Generative.ollama(\n",
    "        api_endpoint=\"http://host.docker.internal:11434\",\n",
    "        model=\"gemma2:2b\"\n",
    "    ),\n",
    "    # ========================================\n",
    "    # END - Ollama setup\n",
    "    # ========================================\n",
    "    # # ========================================\n",
    "    # # For those using Cohere:\n",
    "    # # ========================================\n",
    "    # vectorizer_config=[\n",
    "    #     Configure.NamedVectors.text2vec_cohere(\n",
    "    #         name=\"title\",\n",
    "    #         source_properties=[\"title\"]\n",
    "    #     ),\n",
    "    #     Configure.NamedVectors.text2vec_cohere(\n",
    "    #         name=\"all_text\",\n",
    "    #         source_properties=[\"title\", \"overview\"]\n",
    "    #     ),\n",
    "    # ],\n",
    "    # generative_config=Configure.Generative.cohere(),\n",
    "    # # ========================================\n",
    "    # # END - Cohere setup\n",
    "    # # ========================================\n",
    "    # # ========================================\n",
    "    # # For those using OpenAI:\n",
    "    # # ========================================\n",
    "    # vectorizer_config=[\n",
    "    #     Configure.NamedVectors.text2vec_openai(\n",
    "    #         name=\"title\",\n",
    "    #         source_properties=[\"title\"]\n",
    "    #     ),\n",
    "    #     Configure.NamedVectors.text2vec_openai(\n",
    "    #         name=\"all_text\",\n",
    "    #         source_properties=[\"title\", \"overview\"]\n",
    "    #     ),\n",
    "    # ],\n",
    "    # generative_config=Configure.Generative.openai(),\n",
    "    # # ========================================\n",
    "    # # END - OpenAI setup\n",
    "    # # ========================================\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c3fdad45-8916-4c4a-8405-6f28f3eaee40",
   "metadata": {},
   "source": [
    "> Tip: You can get example collection definitions in our documentation:\n",
    "> - https://weaviate.io/developers/weaviate/manage-data/collections"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bd17a8e9-da25-4619-9e37-0787383dafe1",
   "metadata": {},
   "source": [
    "Was our collection created successfully? Let's take a look"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "cfed74df-84a7-43a6-bd79-960b0681be7e",
   "metadata": {
    "executionCancelledAt": null,
    "executionTime": 48,
    "lastExecutedAt": 1692032185778,
    "lastScheduledRunId": null,
    "lastSuccessfullyExecutedCode": "jprint(client.schema.get(\"Question\"))",
    "outputsMetadata": {
     "0": {
      "height": 616,
      "type": "stream"
     }
    },
    "scrolled": true
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "client.collections.exists(\"Movie\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1ec4752-6428-4db0-b5b3-c36db1b6caaf",
   "metadata": {},
   "source": [
    "### Add data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d809f0b6-d816-448a-9619-b8f660b0eef7",
   "metadata": {},
   "source": [
    "We'll add actual objects (SQL rows) to our data. \n",
    "\n",
    "First, let's build objects to add - and take a look at a couple."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "id": "a4453f38-67ac-4de6-9f3d-decc3527149a",
   "metadata": {
    "executionCancelledAt": null,
    "executionTime": 47,
    "lastExecutedAt": 1692032185826,
    "lastScheduledRunId": null,
    "lastSuccessfullyExecutedCode": "for o in data[:2]:\n    obj_body = {\n        \"question\": o[\"Question\"],\n        \"answer\": o[\"Answer\"],\n    }\n    print(obj_body)",
    "outputsMetadata": {
     "0": {
      "height": 97,
      "type": "stream"
     }
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>title</th>\n",
       "      <th>overview</th>\n",
       "      <th>year</th>\n",
       "      <th>popularity</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>Cinderella</td>\n",
       "      <td>Cinderella has faith her dreams of a better li...</td>\n",
       "      <td>1950</td>\n",
       "      <td>100.819</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>Sunset Boulevard</td>\n",
       "      <td>A hack screenwriter writes a screenplay for a ...</td>\n",
       "      <td>1950</td>\n",
       "      <td>57.740</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>Rashomon</td>\n",
       "      <td>Brimming with action while incisively examinin...</td>\n",
       "      <td>1950</td>\n",
       "      <td>21.011</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>Alice in Wonderland</td>\n",
       "      <td>On a golden afternoon, young Alice follows a W...</td>\n",
       "      <td>1951</td>\n",
       "      <td>75.465</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>Singin' in the Rain</td>\n",
       "      <td>In 1927 Hollywood, a silent film production co...</td>\n",
       "      <td>1952</td>\n",
       "      <td>31.407</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "                 title                                           overview  \\\n",
       "0           Cinderella  Cinderella has faith her dreams of a better li...   \n",
       "1     Sunset Boulevard  A hack screenwriter writes a screenplay for a ...   \n",
       "2             Rashomon  Brimming with action while incisively examinin...   \n",
       "3  Alice in Wonderland  On a golden afternoon, young Alice follows a W...   \n",
       "4  Singin' in the Rain  In 1927 Hollywood, a silent film production co...   \n",
       "\n",
       "   year  popularity  \n",
       "0  1950     100.819  \n",
       "1  1950      57.740  \n",
       "2  1950      21.011  \n",
       "3  1951      75.465  \n",
       "4  1952      31.407  "
      ]
     },
     "execution_count": 42,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data_columns = ['title', 'overview', 'year', 'popularity']\n",
    "\n",
    "df = movie_df[data_columns]\n",
    "\n",
    "df.head()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d0d65333-ce25-4833-92b7-57ca8113bdb2",
   "metadata": {},
   "source": [
    "> If it all looks fine - let's add objects:\n",
    "> - https://weaviate.io/developers/weaviate/manage-data/import"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "id": "1b445597-3655-4606-99e8-8398c50b632d",
   "metadata": {
    "executionCancelledAt": null,
    "executionTime": 6734,
    "lastExecutedAt": 1692032192560,
    "lastScheduledRunId": null,
    "lastSuccessfullyExecutedCode": "with client.batch() as batch:\n    for o in data:\n        obj_body = {\n            \"question\": o[\"Question\"],\n            \"answer\": o[\"Answer\"],\n        }\n        batch.add_data_object(\n            data_object=obj_body,\n            class_name=\"Question\"\n        )",
    "outputsMetadata": {
     "0": {
      "height": 97,
      "type": "stream"
     }
    }
   },
   "outputs": [
    {
     "name": "stderr",
     "output_type": "stream",
     "text": [
      "1322it [00:15, 87.68it/s]\n"
     ]
    }
   ],
   "source": [
    "from tqdm import tqdm\n",
    "\n",
    "movies = client.collections.get(\"Movie\")\n",
    "\n",
    "with movies.batch.fixed_size(200) as batch:\n",
    "    for i, row in tqdm(df.iterrows()):\n",
    "        obj_body = {\n",
    "            c: row[c] for c in data_columns\n",
    "        }\n",
    "        batch.add_object(\n",
    "            properties=obj_body\n",
    "        )"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "216d67ac-f92d-4239-b1cb-8c76ea2cf6ca",
   "metadata": {},
   "source": [
    "#### Confirm data load"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9003b6a9-1d0c-4f31-a3a2-945a7a17dfd8",
   "metadata": {},
   "source": [
    "Do we have data? \n",
    "\n",
    "Let's get an object count"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 44,
   "id": "748cf869-67cd-4b39-b230-23b043217aa3",
   "metadata": {
    "executionCancelledAt": null,
    "executionTime": 48,
    "lastExecutedAt": 1692032192610,
    "lastScheduledRunId": null,
    "lastSuccessfullyExecutedCode": "jprint(client.query.aggregate(\"Question\").with_meta_count().do())",
    "outputsMetadata": {
     "0": {
      "height": 277,
      "type": "stream"
     }
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "1322\n"
     ]
    }
   ],
   "source": [
    "print(len(movies))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1233f9b5-5676-4dc3-bcc8-86df0f86553f",
   "metadata": {},
   "source": [
    "Does the data look right?\n",
    "\n",
    "Let's grab a few objects from Weaviate!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 45,
   "id": "146e3376-9b3d-4cd6-b27f-e835a9f4eb76",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{'title': 'Good Will Hunting', 'overview': 'When professors discover that an aimless janitor is also a math genius, a therapist helps the young man confront the demons that are holding him back.', 'year': 1997, 'popularity': 144.285}\n",
      "{'title': 'The Amazing Spider-Man', 'overview': \"Peter Parker is an outcast high schooler abandoned by his parents as a boy, leaving him to be raised by his Uncle Ben and Aunt May. Like most teenagers, Peter is trying to figure out who he is and how he got to be the person he is today. As Peter discovers a mysterious briefcase that belonged to his father, he begins a quest to understand his parents' disappearance – leading him directly to Oscorp and the lab of Dr. Curt Connors, his father's former partner. As Spider-Man is set on a collision course with Connors' alter ego, The Lizard, Peter will make life-altering choices to use his powers and shape his destiny to become a hero.\", 'year': 2012, 'popularity': 120.385}\n",
      "{'title': 'The Intouchables', 'overview': 'A true story of two men who should never have met – a quadriplegic aristocrat who was injured in a paragliding accident and a young man from the projects.', 'year': 2011, 'popularity': 72.12}\n"
     ]
    }
   ],
   "source": [
    "response = movies.query.fetch_objects(limit=3)\n",
    "for o in response.objects:\n",
    "    print(o.properties)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "663c77a3-b0f2-429b-b2fe-78cef9d6ac40",
   "metadata": {},
   "source": [
    "Let's pause for a second - because we've done a lot!"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c4b1dbad-dfa7-4f88-a5f5-ea6a0f39e413",
   "metadata": {},
   "source": [
    "#### What did we just do?\n",
    "\n",
    "Here is a conceptual diagram"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "436522dc-8cd0-47c4-8286-f839e5a785d4",
   "metadata": {},
   "source": [
    "![img](https://github.com/weaviate-tutorials/intro-workshop/blob/main/images/object_import_process_full.png?raw=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2ac15074-c5f6-48e9-9950-35ee265e14d0",
   "metadata": {},
   "source": [
    "## Step 3: Work with the data"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "38c5e47f-c408-4fc4-83f8-4e133d040b3b",
   "metadata": {},
   "source": [
    "Let's try a few more involved queries"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "dd5740b2-2846-4462-b140-1b1b9c4544ec",
   "metadata": {},
   "source": [
    "### Filtering (similar to WHERE filter in SQL)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b0c640bf-9f35-4d1e-96e5-455b2fb69fde",
   "metadata": {},
   "source": [
    "Let's find objects that meet a particular condition."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 46,
   "id": "db451d4b-fcdf-440f-8502-854d970c98d6",
   "metadata": {
    "executionCancelledAt": null,
    "executionTime": 48,
    "lastExecutedAt": 1692032192706,
    "lastScheduledRunId": null,
    "lastSuccessfullyExecutedCode": "where_filter = {\n    \"path\": [\"question\"],\n    \"operator\": \"Like\",\n    \"valueText\": \"*history*\"\n}\n\nresponse = (\n    client.query\n    .get(\"Question\", [\"question\", \"answer\"])\n    .with_where(where_filter)\n    .with_limit(3)\n    .do()\n)\n\njprint(response)",
    "outputsMetadata": {
     "0": {
      "height": 477,
      "type": "stream"
     }
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Suicide Squad\n",
      "Doctor Strange\n",
      "X-Men: Apocalypse\n"
     ]
    }
   ],
   "source": [
    "from weaviate.classes.query import Filter\n",
    "\n",
    "response = movies.query.fetch_objects(\n",
    "    filters=Filter.by_property(\"year\").greater_than(2015),\n",
    "    limit=3\n",
    ")\n",
    "\n",
    "for o in response.objects:\n",
    "    print(o.properties[\"title\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d4bd3737-068a-4a88-8386-0148999947ed",
   "metadata": {},
   "source": [
    "But this does not rank the result in any meaningful way. \n",
    "\n",
    "For that, we need a keyword search (as opposed to a keyword *filter*)."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6553f176-47b5-4c2b-88e2-5c8fea71fe7e",
   "metadata": {},
   "source": [
    "### Keyword search"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d7a3d7c7-d376-465e-8a23-841203d54055",
   "metadata": {},
   "source": [
    "Unlike a keyword filter, a keyword search will search for, and rank results based on the frequency of the keyword."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "id": "62d78900-199a-4b8d-8da1-3223c3ada047",
   "metadata": {
    "executionCancelledAt": null,
    "executionTime": 52,
    "lastExecutedAt": 1692032192810,
    "lastScheduledRunId": null,
    "lastSuccessfullyExecutedCode": "response = (\n    client.query\n    .get(\"Question\", [\"question\", \"answer\"])\n    .with_bm25(\"history\")\n    .with_limit(3)\n    .do()\n)\n\njprint(response)",
    "outputsMetadata": {
     "0": {
      "height": 477,
      "type": "stream"
     }
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "2.310587167739868\n",
      "2024-09-06 17:38:55.270000+00:00\n",
      "{'title': 'Guardians of the Galaxy Vol. 3', 'overview': 'Peter Quill, still reeling from the loss of Gamora, must rally his team around him to defend the universe along with protecting one of their own. A mission that, if not completed successfully, could quite possibly lead to the end of the Guardians as we know them.', 'year': 2023, 'popularity': 165.416}\n",
      "2.310587167739868\n",
      "2024-09-06 17:38:50.464000+00:00\n",
      "{'year': 2017, 'title': 'Guardians of the Galaxy Vol. 2', 'overview': \"The Guardians must fight to keep their newfound family together as they unravel the mysteries of Peter Quill's true parentage.\", 'popularity': 142.267}\n",
      "2.1399571895599365\n",
      "2024-09-06 17:38:37.638000+00:00\n",
      "{'year': 2002, 'title': 'Star Wars: Episode II - Attack of the Clones', 'overview': 'Following an assassination attempt on Senator Padmé Amidala, Jedi Knights Anakin Skywalker and Obi-Wan Kenobi investigate a mysterious plot that could change the galaxy forever.', 'popularity': 44.575}\n",
      "2.0021612644195557\n",
      "2024-09-06 17:38:50.464000+00:00\n",
      "{'year': 2017, 'title': 'Alien: Covenant', 'overview': 'The crew of the colony ship Covenant, bound for a remote planet on the far side of the galaxy, discovers what they think is an uncharted paradise but is actually a dark, dangerous world.', 'popularity': 235.941}\n",
      "1.8524587154388428\n",
      "2024-09-06 17:38:37.638000+00:00\n",
      "{'year': 1999, 'title': 'Galaxy Quest', 'overview': 'For four years, the courageous crew of the NSEA protector - \"Commander Peter Quincy Taggart\" (Tim Allen), \"Lt. Tawny Madison (Sigourney Weaver) and \"Dr.Lazarus\" (Alan Rickman) - set off on a thrilling and often dangerous mission in space...and then their series was cancelled! Now, twenty years later, aliens under attack have mistaken the Galaxy Quest television transmissions for \"historical documents\" and beam up the crew of has-been actors to save the universe. With no script, no director and no clue, the actors must turn in the performances of their lives.', 'popularity': 62.01}\n"
     ]
    }
   ],
   "source": [
    "from weaviate.classes.query import MetadataQuery\n",
    "\n",
    "response = movies.query.bm25(\n",
    "    query=\"galaxy\",\n",
    "    limit=5,\n",
    "    return_metadata=MetadataQuery(score=True, last_update_time=True)\n",
    ")\n",
    "\n",
    "for o in response.objects:\n",
    "    print(o.metadata.score)\n",
    "    print(o.metadata.last_update_time)\n",
    "    print(o.properties)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ada69268-32aa-4175-99e8-516e355d5703",
   "metadata": {},
   "source": [
    "### Semantic search"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa422c04-ae07-48e4-8202-1ea0615db2a0",
   "metadata": {},
   "source": [
    "A semantic search, on the other hand, searches objects based on similarity"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 48,
   "id": "295bcedc-496d-49a0-b8c1-b3a398d728b8",
   "metadata": {
    "executionCancelledAt": null,
    "executionTime": 159,
    "lastExecutedAt": 1692032192969,
    "lastScheduledRunId": null,
    "lastSuccessfullyExecutedCode": "response = (\n    client.query\n    .get(\"Question\", [\"question\", \"answer\"])\n    .with_near_text({\"concepts\": [\"history\"]})\n    .with_limit(3)\n    .do()\n)\n\njprint(response)",
    "outputsMetadata": {
     "0": {
      "height": 477,
      "type": "stream"
     }
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "{\n",
      "  \"title\": \"Galaxy Quest\",\n",
      "  \"overview\": \"For four years, the courageous crew of the NSEA protector - \\\"Commander Peter Quincy Taggart\\\" (Tim Allen), \\\"Lt. Tawny Madison (Sigourney Weaver) and \\\"Dr.Lazarus\\\" (Alan Rickman) - set off on a thrilling and often dangerous mission in space...and then their series was cancelled! Now, twenty years later, aliens under attack have mistaken the Galaxy Quest television transmissions for \\\"historical documents\\\" and beam up the crew of has-been actors to save the universe. With no script, no director and no clue, the actors must turn in the performances of their lives.\",\n",
      "  \"year\": 1999,\n",
      "  \"popularity\": 62.01\n",
      "}\n",
      "{\n",
      "  \"title\": \"Star Wars\",\n",
      "  \"overview\": \"Princess Leia is captured and held hostage by the evil Imperial forces in their effort to take over the galactic Empire. Venturesome Luke Skywalker and dashing captain Han Solo team together with the loveable robot duo R2-D2 and C-3PO to rescue the beautiful princess and restore peace and justice in the Empire.\",\n",
      "  \"year\": 1977,\n",
      "  \"popularity\": 250.715\n",
      "}\n",
      "{\n",
      "  \"title\": \"Interstellar\",\n",
      "  \"overview\": \"The adventures of a group of explorers who make use of a newly discovered wormhole to surpass the limitations on human space travel and conquer the vast distances involved in an interstellar voyage.\",\n",
      "  \"year\": 2014,\n",
      "  \"popularity\": 283.624\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "\n",
    "response = movies.query.near_text(\n",
    "    query=\"galaxy\",\n",
    "    limit=3,\n",
    "    target_vector=\"title\",\n",
    ")\n",
    "\n",
    "for o in response.objects:\n",
    "    print(json.dumps(o.properties, indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6ab8beb0-ab45-4590-a4de-84b68370c026",
   "metadata": {},
   "source": [
    "#### How does this work?\n",
    "\n",
    "- Under the hood, this uses a vector search. It looks for objects which are the most similar to a text input.\n",
    "- We can inspect the similarity along with the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 49,
   "id": "185a77e4-b27a-4b80-9981-fd975f92e67a",
   "metadata": {
    "executionCancelledAt": null,
    "executionTime": 142,
    "lastExecutedAt": 1692032193111,
    "lastScheduledRunId": null,
    "lastSuccessfullyExecutedCode": "response = (\n    client.query\n    .get(\"Question\", [\"question\", \"answer\"])\n    .with_near_text({\"concepts\": [\"history\"]})\n    .with_additional(\"distance\")\n    .with_limit(3)\n    .do()\n)\n\njprint(response)",
    "outputsMetadata": {
     "0": {
      "height": 616,
      "type": "stream"
     }
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "MetadataReturn(creation_time=None, last_update_time=None, distance=0.2529594898223877, certainty=None, score=None, explain_score=None, is_consistent=None, rerank_score=None)\n",
      "{\n",
      "  \"year\": 1999,\n",
      "  \"title\": \"Galaxy Quest\",\n",
      "  \"overview\": \"For four years, the courageous crew of the NSEA protector - \\\"Commander Peter Quincy Taggart\\\" (Tim Allen), \\\"Lt. Tawny Madison (Sigourney Weaver) and \\\"Dr.Lazarus\\\" (Alan Rickman) - set off on a thrilling and often dangerous mission in space...and then their series was cancelled! Now, twenty years later, aliens under attack have mistaken the Galaxy Quest television transmissions for \\\"historical documents\\\" and beam up the crew of has-been actors to save the universe. With no script, no director and no clue, the actors must turn in the performances of their lives.\",\n",
      "  \"popularity\": 62.01\n",
      "}\n",
      "MetadataReturn(creation_time=None, last_update_time=None, distance=0.4024823307991028, certainty=None, score=None, explain_score=None, is_consistent=None, rerank_score=None)\n",
      "{\n",
      "  \"title\": \"Star Wars\",\n",
      "  \"overview\": \"Princess Leia is captured and held hostage by the evil Imperial forces in their effort to take over the galactic Empire. Venturesome Luke Skywalker and dashing captain Han Solo team together with the loveable robot duo R2-D2 and C-3PO to rescue the beautiful princess and restore peace and justice in the Empire.\",\n",
      "  \"year\": 1977,\n",
      "  \"popularity\": 250.715\n",
      "}\n",
      "MetadataReturn(creation_time=None, last_update_time=None, distance=0.429984450340271, certainty=None, score=None, explain_score=None, is_consistent=None, rerank_score=None)\n",
      "{\n",
      "  \"year\": 2014,\n",
      "  \"title\": \"Interstellar\",\n",
      "  \"overview\": \"The adventures of a group of explorers who make use of a newly discovered wormhole to surpass the limitations on human space travel and conquer the vast distances involved in an interstellar voyage.\",\n",
      "  \"popularity\": 283.624\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "import json\n",
    "\n",
    "response = movies.query.near_text(\n",
    "    query=\"galaxy\",\n",
    "    limit=3,\n",
    "    target_vector=\"title\",\n",
    "    return_metadata=MetadataQuery(distance=True)\n",
    ")\n",
    "\n",
    "for o in response.objects:\n",
    "    print(o.metadata)\n",
    "    print(json.dumps(o.properties, indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6df9e24c-edf7-4f25-ac30-6e291b3ffcad",
   "metadata": {},
   "source": [
    "This is where \"vectors\" come in. \n",
    "\n",
    "Each object in Weaviate includes a vector - like so:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 50,
   "id": "1d75ad29-ce85-4bca-beaf-da708b20fb57",
   "metadata": {
    "executionCancelledAt": null,
    "executionTime": 54,
    "lastExecutedAt": 1692032193165,
    "lastScheduledRunId": null,
    "lastSuccessfullyExecutedCode": "response = (\n    client.query\n    .get(\"Question\", [\"question\", \"answer\"])\n    .with_additional(\"vector\")\n    .with_limit(1)\n    .do()\n)\n\njprint(response)",
    "outputsMetadata": {
     "0": {
      "height": 616,
      "type": "stream"
     }
    },
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "0.2529594898223877\n",
      "{\n",
      "  \"overview\": \"For four years, the courageous crew of the NSEA protector - \\\"Commander Peter Quincy Taggart\\\" (Tim Allen), \\\"Lt. Tawny Madison (Sigourney Weaver) and \\\"Dr.Lazarus\\\" (Alan Rickman) - set off on a thrilling and often dangerous mission in space...and then their series was cancelled! Now, twenty years later, aliens under attack have mistaken the Galaxy Quest television transmissions for \\\"historical documents\\\" and beam up the crew of has-been actors to save the universe. With no script, no director and no clue, the actors must turn in the performances of their lives.\",\n",
      "  \"year\": 1999,\n",
      "  \"title\": \"Galaxy Quest\",\n",
      "  \"popularity\": 62.01\n",
      "}\n",
      "[-0.5893528461456299, 0.5016435980796814, -3.231532096862793, 0.6001119017601013, -0.4437905550003052, 0.6938812732696533, -0.6911576986312866, 0.24620701372623444, -0.40899139642715454, 0.6123637557029724, 0.6818543076515198, 2.7342238426208496, 1.2442903518676758, 0.9821367263793945, 0.15932443737983704, -0.6172680258750916, -0.0966271311044693, -1.030463695526123, -1.2781126499176025, 0.36011025309562683, -0.9227867126464844, -0.23228394985198975, -0.38917824625968933, 0.2675244212150574, 2.2101056575775146, 0.4592687487602234, 0.2656160593032837, -1.3610810041427612, -0.6998542547225952, -0.3802425265312195, 1.5796196460723877, -0.7661688923835754, -0.5394514203071594, -0.05659111961722374, -0.609272301197052, -1.5214133262634277, 0.7674370408058167, 0.44086122512817383, 0.5112622380256653, 1.9179103374481201, -0.9692739844322205, 0.25886908173561096, -0.3246890604496002, 1.3384844064712524, 2.0238540172576904, -0.26892220973968506, 1.1457242965698242, -0.3261510133743286, 1.6416693925857544, -1.0315091609954834, -1.3154888153076172, 0.7665904760360718, -0.3392593562602997, -1.1695616245269775, -1.0172532796859741, 0.6406046152114868, -0.2526523768901825, -1.255115270614624, -0.8330013155937195, -0.4661100208759308, 1.5153021812438965, 0.6550662517547607, 1.271065592765808, 1.5161678791046143, 0.7403848767280579, -1.2566585540771484, -0.3716559112071991, 0.47761332988739014, -1.3490849733352661, -0.36762693524360657, 0.8779399394989014, -0.054019711911678314, -0.14253735542297363, 0.10212744027376175, -0.10405666381120682, 0.06933558732271194, -0.35696470737457275, 1.39705228805542, -1.6028696298599243, 0.39868780970573425, 0.09830726683139801, 0.8838527202606201, -0.08406320214271545, 0.8121174573898315, -0.7535362243652344, 0.11589307337999344, 0.15300853550434113, 0.11563367396593094, -0.5220543742179871, 1.9397705793380737, 0.25632408261299133, 1.1597533226013184, 0.9830033183097839, -1.1746574640274048, -0.6132236123085022, -0.14320145547389984, -1.0456477403640747, 0.06506050378084183, -0.05025532841682434, -0.05260448157787323, 0.3955066502094269, -0.13624168932437897, 0.6394870281219482, -0.025892432779073715, 0.4236818850040436, 0.8251866102218628, -0.14994312822818756, 1.0118331909179688, 0.6568652987480164, -0.08905192464590073, 0.9454818367958069, 1.4744480848312378, 0.07874741405248642, -1.3149597644805908, 0.912401020526886, 0.5908950567245483, 1.924361228942871, -1.2365093231201172, 0.01734437607228756, 1.1587471961975098, 0.7418584227561951, 0.3896853029727936, -0.9054186940193176, 0.5313979387283325, 0.1177905946969986, 0.2029145061969757, -0.8276607394218445, 0.45452821254730225, -0.5550837516784668, 0.6205629706382751, -0.18068712949752808, -1.0711665153503418, -0.21095559000968933, -0.521125316619873, -1.9028517007827759, -0.04803788661956787, -0.4250224232673645, 0.29409053921699524, -1.033235788345337, 0.8771504163742065, 0.9830077886581421, -0.6013548374176025, 1.0438321828842163, -0.4479304254055023, -0.3045163154602051, -1.2001208066940308, -0.4324963390827179, 0.38924795389175415, -1.4198473691940308, -0.8548570275306702, 0.09505806863307953, 0.38070979714393616, 0.36929747462272644, 0.6981423497200012, 0.4504270553588867, -1.6731220483779907, -0.6400507688522339, 1.6639342308044434, 0.6911987066268921, -0.7818529605865479, 1.1905790567398071, -0.43109992146492004, 0.19981510937213898, 0.4166720509529114, -1.0538721084594727, -1.184035301208496, 1.0550073385238647, 0.7742149829864502, 0.008999603800475597, 1.2751439809799194, -0.909083366394043, -1.3447399139404297, -0.5411215424537659, -0.16003109514713287, 0.1600065529346466, -0.2871147394180298, 0.34494900703430176, -1.673866868019104, 0.3283437490463257, -0.18153207004070282, 0.5275561809539795, -0.4878386855125427, 0.24927476048469543, 1.1460965871810913, -0.06114566698670387, 0.17517219483852386, 1.5517117977142334, -1.516624927520752, 0.25883716344833374, -0.18152202665805817, -0.09705297648906708, 0.31058067083358765, -1.000404953956604, -0.17164167761802673, -0.18334762752056122, -0.6708835363388062, 1.3018766641616821, -0.16478130221366882, -0.5595703721046448, -0.47497662901878357, 0.22424235939979553, -0.3446475863456726, -0.42467567324638367, -0.33798539638519287, -0.21515882015228271, -0.3838551640510559, 0.33900555968284607, 0.557445228099823, 0.0006401576101779938, -0.9705424904823303, 1.0516332387924194, -0.7846791744232178, 0.0630110427737236, -1.0100419521331787, -0.13506980240345, -0.7302964925765991, 0.4666144549846649, -1.1651476621627808, -0.06571318954229355, 0.5019713640213013, 0.539250373840332, 0.14255258440971375, -0.687609076499939, -1.0181595087051392, 1.5170111656188965, -0.4490354657173157, -0.7066466808319092, -0.6369689702987671, 0.11617949604988098, 0.6516740322113037, -0.5375462770462036, -2.2085916996002197, -0.11645243316888809, 1.5383033752441406, 1.7390289306640625, 0.477451890707016, -0.13911591470241547, 0.46027544140815735, 0.5896139144897461, -0.3226287066936493, 0.3371199369430542, 0.6711918115615845, -0.06108052656054497, -0.47156214714050293, -0.4626138210296631, 0.45182299613952637, -0.8316468000411987, -0.40430784225463867, 0.394069641828537, 0.3226349651813507, 0.42738449573516846, -0.8289939165115356, -0.04555812478065491, 0.3245185613632202, 0.9068987369537354, -0.552462100982666, -0.7910137176513672, 0.6407718062400818, 0.9472597241401672, -0.3863939642906189, -0.029282264411449432, -0.29224997758865356, 0.4005332291126251, -1.6048035621643066, -0.16331017017364502, -0.6821165084838867, 0.18942642211914062, 1.205901026725769, -0.5067828893661499, -0.8739772439002991, 1.1269629001617432, 0.4962576925754547, -0.20691382884979248, -0.034620024263858795, -0.40717339515686035, -0.35271528363227844, 0.7078899145126343, -0.041619107127189636, 0.3478440046310425, 0.7105185389518738, -0.32144075632095337, -1.9148730039596558, -1.623765468597412, 1.20070481300354, -0.06365875154733658, 0.5907420516014099, 0.19451460242271423, 0.6854471564292908, 0.4696725010871887, 0.40399283170700073, 1.3018076419830322, -1.3212190866470337, -0.7092313766479492, 0.5103670954704285, -0.04615987837314606, -0.06139529123902321, 1.247298240661621, 0.5324232578277588, 0.7877655029296875, -1.3954764604568481, 0.705203652381897, -0.8734809160232544, 0.1343059539794922, 1.1116992235183716, -1.6487489938735962, -0.701229453086853, 1.6215322017669678, -0.1915028691291809, 0.24870353937149048, -0.43895959854125977, 0.17755302786827087, 0.43185511231422424, -0.4500570297241211, 0.9937201738357544, 0.5024891495704651, 0.49539005756378174, -0.1444757729768753, 0.757590651512146, 0.7135387063026428, 0.12213055044412613, -0.9404066801071167, -0.8010443449020386, -1.3299371004104614, -1.490972876548767, -0.1641787886619568, 0.6993786096572876, -0.5026119351387024, -0.24795976281166077, -0.26451778411865234, -0.10924641042947769, -0.7795665264129639, 0.8606106042861938, -0.8164874911308289, -0.616368293762207, 0.17224279046058655, 0.5768897533416748, -0.10647733509540558, -0.162778839468956, -0.6119098663330078, 0.7048877477645874, -0.4909605383872986, -0.3007664680480957, -0.21869349479675293, -0.025044552981853485, -0.9579674005508423, 0.19545672833919525, -0.2767823040485382, 0.5790770053863525, 0.4732905328273773, 0.5141600370407104, -1.2544533014297485, -0.05576815456151962, -0.6983236074447632, -0.15514220297336578, 0.4843885600566864, -0.5650084018707275, 0.008469682186841965, 0.7876850962638855, 1.2364248037338257, -0.1437755823135376, 0.2647949755191803, -1.168021321296692, -0.6634164452552795, 0.7863108515739441, 0.4506657123565674, -0.10490360856056213, -1.0288141965866089, -0.7011650800704956, 1.2436789274215698, 0.3182925581932068, 0.34505128860473633, -0.7568315863609314, -0.322908878326416, 0.3271094262599945, 0.10262523591518402, 0.022371716797351837, -1.9854966402053833, -1.0963417291641235, -0.9945175647735596, 0.44639378786087036, 0.19160406291484833, -0.6370074152946472, 1.6785643100738525, 0.34512490034103394, 0.3960711359977722, -0.8868220448493958, 1.3121883869171143, -0.9871198534965515, 0.011796500533819199, 0.4520544409751892, 0.5981612205505371, -0.7194169759750366, 0.36738836765289307, 0.9273992776870728, 0.22037824988365173, -2.2657811641693115, 1.0250953435897827, -0.11760345101356506, 0.8979575037956238, -0.23571284115314484, -0.6399440169334412, 0.07972157001495361, -0.06769385933876038, 0.2886158227920532, 0.10215103626251221, -0.7824228405952454, -0.6125041246414185, -1.3439275026321411, 0.11447685956954956, 0.3062092661857605, -0.047509487718343735, 0.8233569264411926, 0.6339717507362366, -0.837181568145752, 1.252034306526184, -0.33717745542526245, 0.4821934401988983, -0.5206741094589233, -0.24719908833503723, 0.2412356585264206, 0.8293523788452148, 1.4268263578414917, 0.5742097496986389, 0.3624259829521179, 0.12156014144420624, 0.31361761689186096, -0.9036523103713989, 0.7297161817550659, 0.4446043074131012, -1.212738037109375, -0.7170639038085938, 0.3709704875946045, 1.8544831275939941, 1.3325539827346802, -1.6559199094772339, -0.055497221648693085, 0.24093487858772278, -0.6332218050956726, 1.2391917705535889, 0.6419998407363892, 0.8752851486206055, -1.440816044807434, -0.6726747155189514, 0.9336714744567871, 0.9559942483901978, 2.044456958770752, -0.38469311594963074, -0.3327365517616272, -1.3611263036727905, 0.4046238660812378, -0.007219977676868439, -0.5719611644744873, -0.9789636731147766, 0.565775990486145, 1.1140391826629639, -0.15133602917194366, 1.0024486780166626, 0.9234732389450073, 0.4394773244857788, 0.628991961479187, -0.35276874899864197, 0.8134509325027466, -1.4902982711791992, 0.5884863138198853, 1.7362617254257202, -0.360654354095459, 0.4948638081550598, -0.4390205442905426, 1.1910725831985474, 2.4036505222320557, -0.6459802985191345, -0.35851526260375977, 1.8608442544937134, -0.5327927470207214, -0.9838869571685791, -0.14674724638462067, -1.1613123416900635, 1.0741236209869385, -0.679739773273468, 0.18358169496059418, -0.6163291335105896, -0.49480587244033813, 0.06969122588634491, -1.2263438701629639, 0.06666035950183868, 0.5412982702255249, -0.17288176715373993, -0.2928805649280548, 0.8985770344734192, 0.0311450082808733, 0.38708969950675964, -1.1946390867233276, 0.8775931596755981, -0.5265824794769287, 0.9627684354782104, 0.042379334568977356, -2.486402988433838, -1.365224838256836, -0.20143845677375793, -0.16498121619224548, 0.8019641041755676, 1.1882812976837158, -0.9803305268287659, -1.0458842515945435, -0.5732756853103638, 0.24075858294963837, -0.3198777735233307, -0.8005046844482422, -1.9180023670196533, 1.0676079988479614, -0.6712767481803894, -0.18722723424434662, 1.21397864818573, 0.6203895807266235, 0.6315015554428101, -1.8075155019760132, 1.202556848526001, -0.2962789237499237, 0.7274937629699707, 1.068089246749878, 0.7431405186653137, -1.0176291465759277, 0.5373110175132751, -1.2838276624679565, 0.06291160732507706, -0.7455135583877563, 0.3634178638458252, -0.14686399698257446, 0.8947314023971558, -1.0931135416030884, 1.6324553489685059, -0.010121053084731102, -0.5443814396858215, -1.076090693473816, -0.6425447463989258, -0.30601775646209717, -0.7295992374420166, 0.6647988557815552, 0.21203693747520447, 0.7028579711914062, 0.18534550070762634, 0.27280303835868835, -0.7117269039154053, -0.5323472023010254, -0.3906437158584595, 0.13692937791347504, 0.23610205948352814, -0.3643651306629181, 0.5883479118347168, -0.8434590101242065, 0.5807004570960999, -1.2844829559326172, -0.2725357711315155, -0.2814917266368866, 0.2973988950252533, -0.20171043276786804, 1.151362657546997, -0.6326951384544373, 0.2556583285331726, -0.7954135537147522, 0.09447675943374634, -1.4933019876480103, 0.3008962571620941, -0.25788965821266174, 0.7889779210090637, -0.7157233357429504, -0.16300395131111145, -0.5497468709945679, 0.8704156279563904, -0.1965707689523697, -0.40783917903900146, -0.12696686387062073, -0.8735842704772949, 0.006473630666732788, -0.4162387251853943, -0.8080015778541565, -0.45015382766723633, 0.5542208552360535, -1.0591943264007568, -0.7680675983428955, 0.12791478633880615, -0.25158193707466125, 0.9253849983215332, 0.4500551223754883, 0.32595157623291016, 0.8242647647857666, -1.2037022113800049, -0.028161104768514633, 1.0501291751861572, -1.4166483879089355, -0.1833513230085373, -1.017044186592102, 0.4326711893081665, 0.037024758756160736, -0.4408622980117798, 0.3604497015476227, -0.21616274118423462, -0.16226163506507874, 0.5612316131591797, -1.4924848079681396, -0.5330606698989868, -0.13244490325450897, 0.5753709077835083, -1.0141849517822266, -0.2754591703414917, -0.4976629912853241, -0.22596320509910583, -0.11440599709749222, 1.2530193328857422, -0.4628691077232361, 0.7938877940177917, 0.40845224261283875, 0.06177569553256035, -1.2171849012374878, 1.0818594694137573, 0.054021261632442474, 2.0730068683624268, -0.13122762739658356, 0.4026600122451782, -0.29333609342575073, -0.6585884690284729, -0.8371224403381348, 0.02750599943101406, -0.836267352104187, 0.9901964664459229, -0.6878924369812012, -1.5320261716842651, -0.19961072504520416, -0.11711131036281586, 0.5686215162277222, -0.7617371678352356, 0.9281812906265259, -0.6758561134338379, -0.8183483481407166, -0.45140165090560913, 0.15367817878723145, -1.7312508821487427, -1.0664560794830322, 0.4336293041706085, 0.6199203133583069, 1.3384373188018799, -0.1283494532108307, 0.35641419887542725, 0.1719597578048706, -0.01535089686512947, 1.2813855409622192, -0.28727883100509644, -0.6403345465660095, 1.7518795728683472, -0.9847121238708496, 1.2415655851364136, 0.8239833116531372, 1.4967328310012817, -0.09325681626796722, -0.637978196144104, -0.2261599600315094, 0.7009279727935791, -0.9054114818572998, -1.6855536699295044, 0.4728192687034607, -0.024895820766687393, -0.4336710572242737, -1.4468107223510742, 0.46671682596206665, 1.5811306238174438, 0.09563493728637695, -0.4133668541908264, -0.25475382804870605, -1.3913105726242065, -0.01001480408012867, 0.9143986701965332, 0.4615567922592163, -0.9785907864570618, 0.7117308974266052, -0.05609399080276489, 0.1869291067123413, 0.5312349796295166, 0.9016588926315308, -0.1685260385274887, 0.7842245101928711, -0.4320051670074463, -0.14888910949230194, 0.7225746512413025, 0.6264124512672424, 0.26219576597213745, -0.022230520844459534, 0.7163125872612, -1.4377102851867676, 1.3949763774871826, -1.300102949142456, -1.4881348609924316, -1.043715238571167, 0.6620921492576599, -0.707241952419281, 0.39846113324165344, -1.0561819076538086, 0.2855767607688904, 1.339228868484497, -1.0273594856262207, 0.22380627691745758, 0.08650241792201996, 0.9092939496040344, -0.16757623851299286, 0.0508735217154026, -2.2220044136047363, -0.05272930860519409, 0.9985554218292236, -0.18453270196914673, 1.0091784000396729, 0.7584682106971741, -0.28045082092285156, 0.1447179615497589, -0.49181774258613586, 0.5994261503219604, 0.7970297932624817, 0.47490566968917847, 0.04072338715195656, 0.7103673219680786, 0.8907889723777771, -0.10123927891254425, 1.1596851348876953, -0.17676660418510437, -0.6851291656494141, 0.04090345278382301, -0.24842512607574463, 1.04790198802948, 0.9443440437316895, -1.343252420425415, 0.3716764450073242, -0.08602580428123474, -0.08891081809997559, 1.1139662265777588, -0.5918657779693604, -0.5899898409843445, -0.03595007210969925, -0.19899705052375793, -1.74886155128479, -1.2481064796447754, -0.20874080061912537, -0.6261931657791138, 0.11115044355392456, 0.38015294075012207, -0.26442593336105347, -0.2661326825618744, -0.14804156124591827, 0.31282949447631836, 0.49210602045059204, 1.3384015560150146, 0.3503924310207367, -0.87349534034729, 0.04380808398127556, 0.4091568887233734, 0.7384667992591858, 0.19114729762077332, 0.67444908618927, 0.4946530759334564, 0.5388075113296509, 0.46271049976348877, 2.651381015777588, -0.6947360634803772, 0.35390597581863403, -0.48849961161613464, 0.5268287658691406, 0.24472370743751526, 0.027634523808956146, 0.03421654924750328, -0.3972530663013458, 1.269540548324585]\n",
      "0.4024823307991028\n",
      "{\n",
      "  \"title\": \"Star Wars\",\n",
      "  \"overview\": \"Princess Leia is captured and held hostage by the evil Imperial forces in their effort to take over the galactic Empire. Venturesome Luke Skywalker and dashing captain Han Solo team together with the loveable robot duo R2-D2 and C-3PO to rescue the beautiful princess and restore peace and justice in the Empire.\",\n",
      "  \"year\": 1977,\n",
      "  \"popularity\": 250.715\n",
      "}\n",
      "[-0.9034830927848816, -0.7712558507919312, -3.110103130340576, 0.34885266423225403, -0.9137812256813049, 0.9157955050468445, -0.41015732288360596, 0.5601257085800171, 0.0791725441813469, -0.36848193407058716, 0.5335967540740967, 2.249262571334839, 1.2543363571166992, 0.6753981709480286, 0.8382073640823364, -1.704640507698059, 0.9692229628562927, -1.3826197385787964, -0.5831307172775269, 1.1612465381622314, -0.10671717673540115, -1.2859389781951904, 0.14442887902259827, 0.5419842600822449, 1.998113989830017, 0.06546464562416077, -1.1082563400268555, -1.1271908283233643, -1.2458031177520752, 0.1047295555472374, 2.526864767074585, 0.4967622756958008, -0.6846490502357483, -0.8836025595664978, 0.12007632851600647, -2.1888744831085205, -1.3089746236801147, 0.015501148998737335, 1.510339617729187, -0.08974966406822205, -0.7696452140808105, 0.4316391944885254, 0.9585919380187988, 1.583344578742981, 1.428924560546875, -0.4273514449596405, 0.688667356967926, 0.4245258867740631, 1.0388996601104736, -0.6337156891822815, -0.19296054542064667, 0.14807634055614471, 0.540660560131073, 0.5698923468589783, 0.17267891764640808, -0.3596397638320923, 0.04418633133172989, -1.7904151678085327, -0.4135861396789551, -0.9095456600189209, 1.5649006366729736, 1.250490665435791, 0.5639126896858215, 1.205120325088501, 0.02873702347278595, -1.560034155845642, -0.3573325574398041, 0.6182973384857178, 1.002816915512085, -0.9213179349899292, 0.8634274005889893, -0.3916949927806854, 1.0197921991348267, -1.284744143486023, -1.3258414268493652, 0.15215493738651276, 0.4629869759082794, 0.9976269006729126, -1.7488361597061157, 0.7536839842796326, 0.5815195441246033, -1.1820812225341797, 0.11907132714986801, 1.5915954113006592, 1.002511978149414, 0.5229453444480896, -0.10731145739555359, 0.16298481822013855, -0.026615772396326065, 1.4136254787445068, 0.48948022723197937, 1.3717975616455078, 0.2488292157649994, 0.1926918476819992, -1.2311129570007324, -0.8242984414100647, -0.10669530928134918, 0.18822069466114044, 0.1517588496208191, -1.1176222562789917, -0.22683057188987732, -0.5198184847831726, 1.1753861904144287, -1.4695833921432495, 0.8233999609947205, 0.8511928915977478, -0.22175544500350952, 0.9091070294380188, -0.6910483241081238, -0.594063937664032, -0.20847636461257935, 0.23840577900409698, 0.5295582413673401, -1.091687798500061, 0.45720145106315613, 0.48953989148139954, 1.2342954874038696, -1.1797682046890259, -0.025345994159579277, 0.6703697443008423, 1.1985026597976685, -0.48521074652671814, -0.325394868850708, -0.4361659288406372, 0.12375552207231522, -0.05920146778225899, 0.19466964900493622, -0.29750531911849976, -0.07510055601596832, -1.101442575454712, 0.17774415016174316, -0.35401976108551025, -0.4881007969379425, -0.641503095626831, -1.978952169418335, 0.5628073215484619, -0.509523332118988, -1.3346118927001953, -0.8592608571052551, -0.06602383404970169, 1.2784578800201416, 0.6179754734039307, 1.1152585744857788, -0.4721040427684784, -0.19581656157970428, -0.7950148582458496, -0.051702216267585754, -0.24887312948703766, -0.7386614680290222, -0.3538350462913513, 1.119154930114746, 0.6711720824241638, -1.0062627792358398, 0.6413167119026184, -0.29111039638519287, -1.3608777523040771, -0.5740146040916443, 0.9682158827781677, 0.7980270385742188, -0.4468594193458557, 0.5703487992286682, -1.2797961235046387, 0.4654644727706909, 0.7410905361175537, -0.555234968662262, -0.9370290040969849, 0.04963647946715355, 0.5903587341308594, 0.3345164954662323, 0.6519864797592163, -1.043924331665039, -0.8925063014030457, 0.3871304988861084, -1.1694908142089844, 1.3228325843811035, -0.8659963607788086, -0.9050350189208984, -0.9910804629325867, -0.3051893413066864, 0.07008569687604904, -0.7096827030181885, -1.039882779121399, -0.5493609309196472, 1.4567053318023682, -0.3033455014228821, -0.6831284165382385, -0.05399731174111366, -1.6090621948242188, -0.9146634340286255, 0.26005589962005615, 0.1935899257659912, 0.728377103805542, -0.3719201982021332, -0.7480988502502441, -0.20357611775398254, 0.5515356659889221, 0.9652868509292603, -0.3214612901210785, 0.1713155210018158, -1.280928611755371, 0.02474571019411087, -0.9437530040740967, -0.18198716640472412, 0.4116055369377136, -1.1627715826034546, 0.3748454451560974, 0.5192548036575317, 1.2951931953430176, 0.5800548195838928, -0.31436407566070557, 0.04909568652510643, -0.9145258665084839, -0.29731041193008423, -0.4526214599609375, 0.12758955359458923, -1.2408547401428223, 0.26249608397483826, -0.7278748154640198, -1.4733679294586182, 0.4904308319091797, -0.1405782252550125, 0.11561427265405655, 0.729745626449585, 0.15321724116802216, 1.0279008150100708, 0.05390072241425514, -0.11275502294301987, -0.21093645691871643, 0.04400833696126938, 0.3402608036994934, 0.3591914474964142, -1.170447826385498, 0.7206606268882751, 0.766688883304596, 1.0679144859313965, -0.7041816115379333, 0.5022504329681396, 0.3127908706665039, 0.2933502197265625, -0.3528302311897278, 0.23812751471996307, 0.9122512340545654, 0.24286393821239471, -0.14427703619003296, -1.5603619813919067, 1.1105738878250122, -0.20161288976669312, -0.7899584174156189, 0.2980133593082428, 0.1941564381122589, -0.5983687043190002, 0.0814565122127533, -0.22216859459877014, -0.33518296480178833, 0.8559100031852722, -1.0054645538330078, 0.09587104618549347, 0.33133944869041443, 0.4392819106578827, -0.19930799305438995, 0.44535523653030396, -0.3874354660511017, 0.4613102972507477, -1.6435273885726929, 1.256689429283142, 1.039193034172058, -0.4842686057090759, 0.8740160465240479, 0.7925766110420227, -0.824455976486206, 0.4094921350479126, 0.7310092449188232, -0.5137954950332642, -0.4006079435348511, -0.413831502199173, 0.3327244520187378, 0.8113318681716919, 0.00973384827375412, -0.6312618851661682, 0.2339329570531845, -0.4086371064186096, -0.1090383529663086, -1.4330650568008423, 0.6350550055503845, -0.4946949779987335, -0.01890711858868599, 1.1086664199829102, 0.6947574615478516, -0.05670914053916931, -0.4541912376880646, 0.45487695932388306, 0.040366195142269135, -1.3213632106781006, -0.8478949069976807, 0.6297876238822937, 0.10263507068157196, 1.8823142051696777, 0.765659749507904, -0.004136096686124802, -0.31012871861457825, 0.4083876311779022, -0.6087855696678162, 0.9413390159606934, 0.013951979577541351, -1.5522544384002686, -1.2619808912277222, 1.0044704675674438, -0.04014553129673004, 0.1526566743850708, -0.6291879415512085, 0.8087493181228638, 1.5982528924942017, -0.812271773815155, 1.671984314918518, -0.2642384171485901, 1.0959757566452026, 0.5207042098045349, -0.14907270669937134, 1.860628366470337, 0.279957115650177, -0.061440274119377136, -0.537238597869873, -1.4213998317718506, 0.11730853468179703, -0.7179993391036987, 0.6448275446891785, 0.2529550790786743, 0.5816729664802551, -0.04068987816572189, -0.937887966632843, -0.35280561447143555, 0.7404394149780273, 1.2028099298477173, -0.636562705039978, -0.8912844061851501, 0.18243730068206787, -0.5878967046737671, 0.3174334168434143, -0.5964304208755493, 1.1908299922943115, -0.13087502121925354, 0.18885909020900726, 0.005004964768886566, -0.7977849245071411, -0.9573194980621338, 0.08712709695100784, 1.0396438837051392, -0.07296095788478851, 0.3684183657169342, -0.08623526990413666, -1.2983624935150146, -0.5493127703666687, -0.7274203896522522, -0.41067180037498474, -0.33120205998420715, 0.09043972194194794, 0.6877098083496094, 1.8900701999664307, 0.7406724095344543, -0.876969039440155, -0.2873999774456024, -0.8867628574371338, 0.27650338411331177, -0.17663422226905823, -0.3843589127063751, -0.07660505920648575, 0.27615076303482056, -0.4810088276863098, 0.7666136026382446, -0.02717338688671589, 0.3474355638027191, 0.4217577576637268, 0.3751786947250366, 0.9275217652320862, -0.31196266412734985, -0.028005655854940414, -0.5152560472488403, -0.4399161636829376, -0.2010369747877121, 0.6702531576156616, 1.3701053857803345, 0.4478522539138794, 0.18008747696876526, 0.8217387795448303, 0.6551491618156433, -1.679417371749878, 0.2781076729297638, -1.0331817865371704, 0.43891721963882446, 0.38695740699768066, -0.3893444538116455, -1.4909244775772095, 0.777140736579895, -0.10729872435331345, 0.42520296573638916, -2.2155988216400146, 1.2287882566452026, 0.14590305089950562, -0.028547123074531555, -0.7664620876312256, -0.6898770332336426, -0.06055596098303795, 0.7518354058265686, -0.13794997334480286, 0.2653142809867859, 0.43439966440200806, -0.9244863390922546, -0.49381738901138306, -0.4958755075931549, 0.4457762837409973, -0.21486784517765045, -0.12530794739723206, 0.4276111125946045, -0.8455788493156433, 0.7520278096199036, 0.8389731049537659, 1.372009038925171, 1.5874866247177124, -0.009937822818756104, -0.05242006108164787, 0.70148766040802, 1.4294254779815674, 0.057006049901247025, 0.1501670777797699, 0.08425311744213104, 0.8896566033363342, -0.7476268410682678, 0.26628026366233826, 0.2647272050380707, -1.4138908386230469, -0.3366978168487549, 0.2204827219247818, 0.9134024381637573, 1.2315541505813599, -1.0556964874267578, 1.237622618675232, -0.09678009152412415, -0.5805639624595642, 0.1914413422346115, -0.7104465365409851, 0.6398401260375977, -1.13736891746521, -0.8197296857833862, 0.6315931081771851, 0.6232782006263733, 1.7901999950408936, 0.4590883255004883, -1.050988793373108, -1.7986477613449097, -0.6556549668312073, 0.20182731747627258, -0.38777485489845276, 0.6002063751220703, 0.6135458946228027, 1.4512560367584229, -0.36776095628738403, -0.5384587049484253, 1.0232166051864624, 0.058966610580682755, 1.6683686971664429, 0.9561470150947571, 0.3057776689529419, -1.3335764408111572, 0.6862825155258179, 0.9009333252906799, -0.7388688921928406, 0.41684871912002563, 0.09806162863969803, 1.3633604049682617, 2.097869634628296, -0.6371805667877197, -0.00838705338537693, 0.19701722264289856, 0.749299943447113, -0.9345443248748779, -0.3750808537006378, -0.44496554136276245, 0.20430654287338257, 0.04035385325551033, 0.44579240679740906, -0.723305881023407, -0.7054135799407959, -0.8776082396507263, -0.10840655863285065, 0.5709253549575806, 0.2147296965122223, 1.0308018922805786, -0.19424334168434143, 0.9839546084403992, 0.1998952031135559, -0.09670991450548172, 0.5423476696014404, 0.9491016268730164, -0.4541856646537781, 0.5423636436462402, 0.19520099461078644, -1.5371358394622803, -0.6677982807159424, -0.7376106381416321, 1.780181884765625, 0.8327350616455078, 0.21276581287384033, -0.062169551849365234, -1.0132949352264404, 0.012763413600623608, -0.028936825692653656, -1.2387244701385498, -0.7110760807991028, -1.958358645439148, 1.8980114459991455, -0.6873489022254944, 0.39564958214759827, 0.7654470205307007, 0.9162050485610962, 0.26290518045425415, -1.3023560047149658, 1.6511536836624146, -0.11099402606487274, -1.0193274021148682, 0.8180654048919678, 0.6458595991134644, -0.9967106580734253, -0.9396317601203918, -0.32213568687438965, -0.08193626999855042, 0.7240344285964966, 0.5259396433830261, 0.10988926142454147, -0.7810764312744141, -0.7573434710502625, 0.017874881625175476, -0.004837539978325367, 0.05828274041414261, -1.4488662481307983, -0.3061298131942749, -0.23147624731063843, -0.5284687280654907, 0.911439061164856, 0.2756251096725464, 0.43967464566230774, 0.3286304473876953, 0.03481375426054001, -0.4467129111289978, -0.8767462968826294, -0.3784467279911041, -0.18555772304534912, -0.16041098535060883, -0.05241949483752251, 0.19912131130695343, -1.0940459966659546, 0.2741568088531494, -0.5317978262901306, -1.0883872509002686, -0.235260471701622, 0.6427338123321533, -0.12410227954387665, 0.3662703335285187, -0.9390865564346313, 1.047312617301941, -0.9761060476303101, -0.12934574484825134, -0.38676294684410095, 0.448422908782959, 0.2479284405708313, 0.1825442910194397, -0.23762330412864685, -0.2196086049079895, -0.9197337627410889, 0.795425295829773, 0.7698307037353516, -0.3775291442871094, -0.025388410314917564, -0.824148952960968, 0.9033104777336121, -0.10768100619316101, -1.1152350902557373, 0.7164148688316345, 0.8328986167907715, 0.20531126856803894, -0.10692363977432251, 1.2375714778900146, -1.1264078617095947, -0.17254754900932312, 0.13266903162002563, 0.10590648651123047, -0.6246856451034546, -0.5999671220779419, 0.6389572620391846, 0.33584433794021606, -1.1907894611358643, -0.3128967881202698, -1.739493727684021, 0.35082679986953735, 0.4702588617801666, 0.3997584879398346, 0.9142064452171326, -0.5027351379394531, 0.15154479444026947, 0.819205105304718, -1.223906397819519, -1.1239968538284302, 0.4068077802658081, 0.7126778364181519, -0.737285315990448, -0.13631731271743774, -0.5632032752037048, -1.6052366495132446, -0.47104671597480774, 0.6659019589424133, -0.42482954263687134, 0.8567923307418823, -0.43740010261535645, 0.37809044122695923, 0.2522067427635193, 1.1091772317886353, 0.499220073223114, 1.6333028078079224, 0.08285161107778549, -0.7511981725692749, -0.006766706705093384, -0.4301269054412842, -1.223935604095459, 0.8734804391860962, 0.004928648471832275, 0.6866338849067688, -0.5999177098274231, -0.7665576338768005, -0.2365463674068451, -1.3441780805587769, 0.5818890929222107, -0.8423606753349304, 0.21218951046466827, -1.65895414352417, -1.1733214855194092, -0.259297639131546, -0.34799137711524963, -0.7029412984848022, 0.2951308488845825, 0.24408401548862457, 1.1528648138046265, 0.15539750456809998, 0.09455813467502594, -0.4426209032535553, 0.5440112948417664, -0.49026474356651306, 1.5477676391601562, 1.0518265962600708, 0.03746795281767845, 1.1829429864883423, -1.6505699157714844, 1.1091012954711914, 0.6903770565986633, 1.0798369646072388, -0.2250741422176361, -0.6131463646888733, 0.8517099618911743, 1.788141131401062, -1.4813721179962158, -1.002685546875, 0.6923208832740784, 0.3838776648044586, -0.7953752279281616, -3.262023448944092, 1.1257270574569702, 0.5798807144165039, 0.43278416991233826, 0.35880228877067566, 0.2979065477848053, -1.081314206123352, 0.42948901653289795, 0.8188648223876953, -0.36257046461105347, -0.4804147481918335, -0.2549114525318146, -0.22187677025794983, 0.33728379011154175, 0.6262634992599487, 0.7573956847190857, 1.0796890258789062, 1.1298019886016846, -1.2252837419509888, 0.5904399752616882, 0.031114522367715836, 0.7747055888175964, -0.026805702596902847, 0.6922603845596313, 0.9282603859901428, -1.5232454538345337, 1.3027864694595337, -1.8726110458374023, -1.0416022539138794, 0.3328784704208374, -0.23836319148540497, -0.6436648964881897, -0.26896318793296814, -0.9196710586547852, -0.19054101407527924, 0.9891681671142578, -0.4134557545185089, 0.2922303080558777, -0.12516728043556213, 1.114250898361206, -0.5401416420936584, -0.46325188875198364, -2.3938190937042236, -0.2768261134624481, 0.03242398425936699, -0.45484280586242676, 0.013031266629695892, 0.5959180593490601, 1.095766544342041, -0.00940646231174469, 1.0324198007583618, 0.1579093337059021, 0.07685413211584091, 0.42405253648757935, -0.0611988827586174, 0.31034332513809204, -0.36243969202041626, 0.6906462907791138, 1.1595864295959473, 0.699153482913971, -0.6252082586288452, -0.19108068943023682, -0.35938170552253723, 1.0137426853179932, 1.1378556489944458, -1.134177803993225, 0.8990108370780945, -0.19066700339317322, 0.20174676179885864, 0.28150349855422974, -0.5419871211051941, -0.2485431730747223, -0.08438226580619812, 0.7763875126838684, -0.9133841395378113, -0.3938159942626953, -2.018252372741699, 0.156545028090477, 0.13464239239692688, -0.5477105975151062, 0.7858071327209473, 0.006029423326253891, -0.10062593221664429, -0.4224902093410492, 1.1063987016677856, 0.8088204264640808, 0.689007580280304, 0.011448021978139877, 0.18671222031116486, 0.10596436262130737, -0.20354074239730835, -0.15551966428756714, 0.6775224208831787, 0.6173595190048218, -0.2631239593029022, 0.00894222967326641, 1.7056748867034912, -1.206246256828308, 0.06549479067325592, -0.24873468279838562, -0.1550445258617401, 0.08925417065620422, -0.7581641674041748, -0.6031448841094971, -1.0165349245071411, 0.2015334814786911]\n",
      "0.429984450340271\n",
      "{\n",
      "  \"title\": \"Interstellar\",\n",
      "  \"overview\": \"The adventures of a group of explorers who make use of a newly discovered wormhole to surpass the limitations on human space travel and conquer the vast distances involved in an interstellar voyage.\",\n",
      "  \"year\": 2014,\n",
      "  \"popularity\": 283.624\n",
      "}\n",
      "[0.024126943200826645, -0.013838309794664383, -2.8055012226104736, -0.14428117871284485, -0.09647063910961151, 1.3496243953704834, -1.0738040208816528, 0.7684339284896851, -0.711624801158905, -0.7068792581558228, 0.43283918499946594, 1.705535888671875, 0.244768887758255, 1.161515235900879, 0.4575716555118561, -1.3544989824295044, 1.6625181436538696, -1.8636016845703125, 0.4742843508720398, 0.43584999442100525, -0.40639543533325195, -1.4896528720855713, -0.20166262984275818, 1.152583360671997, 1.7474666833877563, 1.845024585723877, 0.5806971192359924, -1.91635000705719, -0.41806119680404663, 0.04038830101490021, 2.138329267501831, -0.905482292175293, -0.18956421315670013, -0.2654317617416382, -0.40517881512641907, -1.9599748849868774, -0.38277193903923035, 1.548977017402649, 0.19338655471801758, 0.34995174407958984, -1.4077081680297852, -0.42695438861846924, 0.4048183858394623, -0.12085622549057007, 2.0703563690185547, 1.4425781965255737, 0.8169280886650085, -0.36945754289627075, 0.3896896243095398, -0.28472474217414856, -1.0798885822296143, 0.56122225522995, -1.714877724647522, -0.2423357367515564, 0.5158200860023499, 0.30540749430656433, -1.0195165872573853, -0.80089271068573, -0.015247346833348274, 0.30288875102996826, 0.37498632073402405, 0.41774123907089233, 1.488635540008545, 1.834883689880371, 0.2473706752061844, -1.6388208866119385, -0.4429314434528351, 1.0251953601837158, 0.19515666365623474, -1.5662257671356201, 0.37981218099594116, -0.9087324142456055, 0.7706607580184937, -0.5606481432914734, -1.5536786317825317, 0.9542458653450012, -0.03690864145755768, 1.286834478378296, -0.21369963884353638, 0.6931717991828918, -0.31995731592178345, -0.622636079788208, -0.12011991441249847, 0.6106230020523071, 0.6507745385169983, 0.36321181058883667, 0.7295473217964172, -0.31962695717811584, -0.44396257400512695, 1.3658721446990967, 0.7596655488014221, 0.4798867702484131, -0.31607329845428467, -0.04974491149187088, -0.528312087059021, -0.4114218056201935, -0.7350025177001953, -0.20753085613250732, -0.2507694363594055, -1.3783061504364014, -0.46315184235572815, -0.5443325042724609, 1.309755802154541, -0.6158835887908936, 1.4882583618164062, 0.5237366557121277, -0.17519716918468475, -0.21881021559238434, -0.5741533637046814, -0.5670974850654602, 0.07726529240608215, -0.1180526465177536, 0.001199150923639536, -1.8693666458129883, 0.5800559520721436, 0.3372580409049988, 0.019267763942480087, -1.0376744270324707, 0.8519923686981201, 1.4110369682312012, 0.4022599458694458, -0.21410900354385376, 0.6974972486495972, -1.0794731378555298, -0.4886276125907898, 0.13242335617542267, -1.159304141998291, 0.43086785078048706, -0.7502848505973816, -0.08240656554698944, -0.07468917220830917, -0.7114626169204712, -0.5688536167144775, -0.6718451380729675, -1.912870168685913, -1.0481033325195312, -0.6004918813705444, -1.02345871925354, -0.9193531274795532, 0.6793264746665955, 1.007086992263794, -0.1807972490787506, 0.6792687177658081, -0.038134776055812836, -0.9195365309715271, -1.5119376182556152, 0.31726962327957153, -0.46310222148895264, -0.44669538736343384, 0.1471964716911316, 0.7606620788574219, 1.066815733909607, 0.48515117168426514, 0.6959579586982727, -0.24275848269462585, -0.903996467590332, 0.5206382870674133, 0.3451467752456665, 0.5325026512145996, -1.1855859756469727, 1.1135321855545044, -0.5520293712615967, 0.23234710097312927, 1.3178030252456665, -1.118688941001892, -1.1093353033065796, 0.3362641930580139, 0.21716231107711792, 0.630100667476654, 0.6311920881271362, -0.954528272151947, -1.1800968647003174, 0.37445950508117676, 0.004550624638795853, 0.8892455697059631, -0.7450593709945679, 0.46711716055870056, -1.8268609046936035, -0.0794815868139267, -0.06714379787445068, -0.06565124541521072, -1.2489911317825317, 0.20275813341140747, 1.4010406732559204, -0.08477997034788132, -0.8060829639434814, -0.26426392793655396, -0.4955810308456421, -1.308514952659607, 0.13137617707252502, 0.39329826831817627, 0.5447916984558105, 0.4204519987106323, -0.7312275171279907, 0.3980805277824402, 0.28660842776298523, 1.8497815132141113, 0.04422437399625778, 0.2521267235279083, -0.46220138669013977, -1.3831279277801514, 0.32764574885368347, -0.10952640324831009, 0.258294939994812, -0.6239141821861267, 0.8060173392295837, 0.3057786226272583, 0.319547176361084, -0.4887516498565674, 0.9232475161552429, 1.0916597843170166, 0.07480680197477341, -0.2833058834075928, -0.07349968701601028, 0.01832319051027298, 0.2481183260679245, 0.1569603681564331, -0.9137770533561707, -0.43858060240745544, -0.05921882763504982, 0.2977932393550873, 0.8244231939315796, 0.8737438917160034, -1.5404748916625977, 1.8331645727157593, -1.3273760080337524, -0.9900192022323608, 0.06874504685401917, -0.07849006354808807, 0.5191847085952759, 0.45632266998291016, -1.737712025642395, -0.2675618827342987, 1.366769552230835, 0.7145166993141174, 0.8612970113754272, -0.2748209834098816, 0.6322439312934875, -0.35477960109710693, -0.1247539222240448, -0.7590895891189575, 0.6543183326721191, -0.4477829039096832, -0.44495218992233276, -1.5431108474731445, -0.09886038303375244, -0.9976597428321838, -2.026993989944458, -0.8805315494537354, 0.4235685169696808, 0.5555522441864014, -0.9202355742454529, -0.8068578243255615, 0.47870302200317383, 0.36504489183425903, -0.5585335493087769, -0.3877509832382202, 1.0848057270050049, -0.25666117668151855, -0.9189946055412292, 0.9461134672164917, -1.1641764640808105, 0.15177568793296814, -2.180601119995117, -0.6677246689796448, -0.6790154576301575, 1.2915006875991821, 1.2450350522994995, 1.0592454671859741, 0.47950395941734314, 0.40594595670700073, 0.19081684947013855, -0.9630427956581116, 0.5178408026695251, -0.16959625482559204, 0.637728214263916, -0.1730809360742569, -0.8321971893310547, -0.5044152736663818, 0.13498438894748688, -0.624936044216156, 0.33213651180267334, -1.9147522449493408, 0.6321308612823486, 0.11587145924568176, -0.603102445602417, 0.5239940285682678, 1.8288202285766602, 0.16955310106277466, 0.11314129829406738, -0.010280869901180267, -1.004289150238037, -0.5120671391487122, 0.8983086347579956, -0.15511633455753326, -0.13982437551021576, 1.5033423900604248, 1.2325530052185059, 0.0029545538127422333, -0.8744305372238159, -0.4808325469493866, -0.5458305478096008, 1.0718882083892822, 0.2073957622051239, -1.0438083410263062, -1.3721672296524048, 0.3414117395877838, 0.3092280626296997, -0.7563287019729614, -1.2183432579040527, 1.4674363136291504, 0.218881294131279, -0.3809907138347626, 1.400895595550537, -0.258014440536499, 0.17226767539978027, 0.9442059397697449, 0.3880913257598877, 1.7588963508605957, 1.4515875577926636, 0.6668091416358948, -0.09924139827489853, -0.6001240611076355, -1.0648094415664673, 0.658164381980896, -0.07191964238882065, 0.02600330114364624, -0.5082076191902161, -0.46456170082092285, -0.23496007919311523, -1.2123222351074219, 0.05095118284225464, 1.691393256187439, -0.3889872133731842, -0.6543072462081909, 0.04510970041155815, -0.8154860734939575, 0.3337303400039673, -0.7847521901130676, 1.0101876258850098, 0.393424928188324, 0.6433790326118469, 0.5913288593292236, -0.2593480050563812, -1.4307610988616943, 0.27005699276924133, 0.6843114495277405, 0.48949727416038513, -0.21252721548080444, 1.1337895393371582, -0.1703210175037384, 0.608571469783783, 0.3463565707206726, -0.12880772352218628, -0.2681373655796051, -0.8496158123016357, 0.7495707869529724, 2.522508144378662, 0.9924014806747437, 0.2064572125673294, -0.5553048849105835, -0.5212310552597046, -0.4034120440483093, -0.4718886613845825, 0.0014713308773934841, -0.3555753231048584, -0.6022216081619263, -0.7107835412025452, 0.7143866419792175, 0.5387625694274902, -0.812091588973999, 0.559719979763031, -0.5259684324264526, 0.645738422870636, -0.2164652943611145, -1.1907345056533813, -0.0074525922536849976, -0.9165112972259521, 0.15999607741832733, 0.8264452219009399, 1.1707674264907837, -0.6576428413391113, 0.25177255272865295, 0.3310126066207886, 0.6358445882797241, -0.09781913459300995, 0.05342693626880646, -0.9601283073425293, -0.008217014372348785, 0.16411098837852478, 1.3884574174880981, -1.7793259620666504, 0.33159059286117554, -0.33397647738456726, -0.4302883446216583, -1.209295392036438, 0.932853102684021, -0.5117630958557129, -0.1944619119167328, -0.6864299178123474, -0.966503918170929, -1.035687804222107, 0.03987761586904526, -0.37198272347450256, 0.16803540289402008, -0.2524259090423584, -0.6511983275413513, -0.9375816583633423, 0.5409572124481201, 1.5134222507476807, 0.2939263582229614, 0.3090224862098694, 0.09080487489700317, -1.0838760137557983, -0.12465158104896545, 0.1262398064136505, 0.7899906635284424, 0.9620431661605835, -0.6659829616546631, 0.3560907542705536, 0.2824987769126892, 1.5630842447280884, 0.13666066527366638, 0.06968600302934647, -0.23108112812042236, -0.021404020488262177, 0.2954292595386505, 0.44489312171936035, -0.5986183881759644, -0.9288169741630554, -1.2846986055374146, -0.12609288096427917, 0.4394773542881012, 1.0530472993850708, -0.9081599712371826, 0.24536824226379395, -0.5577907562255859, -0.13858547806739807, 0.5727635025978088, 0.29361674189567566, 0.5278365015983582, -0.8226278424263, 0.300690233707428, 1.1130192279815674, 0.4222814738750458, 2.5643372535705566, -0.02121138758957386, -1.0143274068832397, -1.079550862312317, 0.28607022762298584, 0.13228751718997955, -0.2583797574043274, -0.746862530708313, 1.030280351638794, 1.5212939977645874, 0.07749342173337936, -0.3651365041732788, 0.4789767861366272, 0.3323844075202942, -0.1660449504852295, 0.26860761642456055, 1.1171973943710327, -0.42770639061927795, 0.2140297144651413, 1.4715330600738525, 0.8523918390274048, -0.31640130281448364, -0.638641357421875, 0.4412226974964142, 1.7932173013687134, 0.00625794380903244, 0.45000362396240234, -0.16750459372997284, -1.043326497077942, -0.9793589115142822, 0.1891772300004959, -0.7683145999908447, -0.18418937921524048, -0.8275027275085449, -0.33357682824134827, -0.14838433265686035, -0.47866272926330566, 0.5481896996498108, 0.05690322071313858, 0.8885924816131592, 0.4284621775150299, 0.9169159531593323, 0.12196752429008484, 1.461764931678772, -0.18204015493392944, 0.37439852952957153, 0.5343922972679138, 0.7581775784492493, -0.16027596592903137, 0.19583743810653687, 0.5412667393684387, -0.5563605427742004, -1.2056525945663452, -0.5222853422164917, 0.6874668002128601, 0.777995765209198, 0.15648522973060608, 0.17719271779060364, -0.07935694605112076, -0.4385080337524414, -0.03686530515551567, 0.6754559278488159, -0.5705336332321167, -1.579899549484253, 1.714098334312439, -1.077040433883667, -0.376033216714859, 0.21632826328277588, 0.38930127024650574, 1.0829371213912964, -0.7625668048858643, 1.8224704265594482, -0.42602869868278503, 0.26921507716178894, -0.38227003812789917, 0.6281564831733704, -0.5323387384414673, -0.16558851301670074, -0.11523298919200897, 0.048530012369155884, 0.7429516315460205, 0.07041936367750168, -0.5662422776222229, 0.3458649814128876, -0.6303927898406982, 0.7131860256195068, -0.21626931428909302, 0.17386029660701752, -0.8683016896247864, 0.6934239864349365, -0.34081342816352844, -0.5480615496635437, 1.5491153001785278, 0.5328972935676575, 1.5499203205108643, -0.12213233858346939, -0.387072890996933, -1.1456173658370972, -0.4878429174423218, 0.1270434856414795, -0.4737698435783386, -0.43622180819511414, 1.087374210357666, 1.093888521194458, -0.12120740115642548, 0.15376442670822144, -1.6226325035095215, 0.8893906474113464, -0.6980355978012085, 1.3095155954360962, -0.1111312061548233, 0.23515988886356354, -0.6403146982192993, 0.09427454322576523, 1.1759088039398193, 0.35689324140548706, -0.5442996025085449, 0.9232907295227051, 0.7211287021636963, -0.09026278555393219, 0.26660865545272827, -0.04166620224714279, -0.6362831592559814, 0.5491138696670532, -0.40874022245407104, -0.19148588180541992, 0.17797733843326569, -0.8107484579086304, -0.7363681793212891, 0.9610925316810608, -0.5932472348213196, 0.8516550660133362, 0.7896484136581421, -0.8017798066139221, -0.38736578822135925, 0.46297475695610046, -1.6368539333343506, 0.8724328279495239, 0.5869635343551636, -0.6239081621170044, -0.17318874597549438, -1.0432674884796143, -0.0622919425368309, 1.9306676387786865, -1.7124532461166382, -0.21685989201068878, -1.2131727933883667, 0.6949489116668701, -0.4364442229270935, -0.7694201469421387, 0.8808854222297668, -1.5573769807815552, 0.08400923013687134, 0.09856196492910385, -0.4448277950286865, -0.15189574658870697, 0.18788956105709076, 1.1171467304229736, -0.6709017157554626, -0.16900117695331573, 0.06361141800880432, -0.6357465386390686, 0.1723923236131668, -0.02368626557290554, 0.293329656124115, -0.012879302725195885, 0.775088906288147, -0.8904247879981995, -1.618107557296753, 0.06232833489775658, -0.3087024688720703, 1.100760579109192, -0.07348328083753586, -0.10331053286790848, -1.0995477437973022, -0.1372833549976349, -1.648991584777832, 0.7350688576698303, -0.9811877608299255, 0.6149348020553589, -1.4546189308166504, -1.050062656402588, -0.47825130820274353, -0.49569734930992126, 1.0486031770706177, -1.1674288511276245, 0.3307546079158783, -0.5370383858680725, -0.28494563698768616, -0.630103588104248, -0.6700348258018494, -1.4789820909500122, 0.07759331166744232, 0.503785252571106, 2.046351671218872, 0.4051980972290039, -0.24084796011447906, -1.2767770290374756, 0.1225213035941124, -0.5304100513458252, 0.8245754837989807, 0.7274953722953796, 0.47538694739341736, 0.6449744701385498, -1.776782751083374, 0.4419146776199341, 1.0307637453079224, 1.3768904209136963, 0.1831303834915161, -0.2991989850997925, 0.23821473121643066, 0.48106706142425537, -1.1130374670028687, -0.08898680657148361, 0.42111286520957947, 0.9483516216278076, -0.6965870261192322, -1.726991057395935, 0.6526636481285095, -0.0017940225079655647, -0.09503549337387085, 0.28976455330848694, 0.10363409668207169, -0.22956721484661102, 0.26860782504081726, 0.3761165142059326, 0.06520574539899826, -1.5975596904754639, -0.17602460086345673, -0.15243683755397797, 1.2168972492218018, 1.0506588220596313, 0.9677528738975525, 0.7140341401100159, 0.639565110206604, 0.4388422966003418, 1.963357925415039, 0.9158220291137695, 0.6945013403892517, -1.028662085533142, 0.2531421184539795, 0.48121872544288635, -1.8071602582931519, 0.282774418592453, -1.796470284461975, -0.6779568195343018, -0.357029527425766, 0.33168792724609375, -0.36806797981262207, 0.06940402835607529, -0.05248863995075226, 0.848025918006897, 0.7416369915008545, -1.431789755821228, 0.5616568326950073, -0.4790877103805542, 0.6330925226211548, 0.3395858407020569, 0.1925625056028366, -1.7586041688919067, -0.7430223226547241, -0.9113904237747192, -0.2120320349931717, 1.1642903089523315, 0.12987878918647766, 0.5393301844596863, -0.8825930953025818, 0.3967943489551544, 1.5188990831375122, 0.35006505250930786, -0.08866339921951294, 0.3597794771194458, 0.30593761801719666, 0.23440489172935486, 0.9283022284507751, 1.324210524559021, 0.31214311718940735, -0.07106472551822662, -0.4585454761981964, -0.7889096736907959, 0.4682426452636719, 0.917502760887146, -1.3422523736953735, 0.21251332759857178, -1.005295753479004, 0.014728579670190811, 0.8569803237915039, -0.8391391038894653, -0.2191655933856964, -0.533972442150116, 0.22870047390460968, -0.1560972034931183, -1.3039393424987793, -1.3251326084136963, 0.21924909949302673, -0.09590344876050949, -0.35198909044265747, -0.5632368326187134, 0.10841101408004761, 0.2175530046224594, -0.15768778324127197, 0.9997344613075256, -0.11957336217164993, 0.316700279712677, 0.46378427743911743, 0.5318368077278137, 0.10190331935882568, -0.4508350193500519, 0.9524921178817749, 1.8278547525405884, -0.3996247947216034, 0.24305963516235352, 0.22375330328941345, 2.0826807022094727, -0.5114635825157166, 0.4034169018268585, -0.2098536193370819, -0.13468575477600098, -0.18663832545280457, 0.13397085666656494, 0.49230653047561646, -0.7607913613319397, 0.5756995677947998]\n"
     ]
    }
   ],
   "source": [
    "response = movies.query.near_text(\n",
    "    query=\"galaxy\",\n",
    "    limit=3,\n",
    "    target_vector=\"title\",  # or \"overview\"\n",
    "    include_vector=True,\n",
    "    return_metadata=MetadataQuery(distance=True)\n",
    ")\n",
    "\n",
    "for o in response.objects:\n",
    "    print(o.metadata.distance)\n",
    "    print(json.dumps(o.properties, indent=2))\n",
    "    print(o.vector[\"title\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4097db1-76b0-459c-b5c3-2ee1e0f90abe",
   "metadata": {},
   "source": [
    "These vector representations come from deep learning models to those that power LLMs. They capture meaning, and are called vector \"embeddings\"."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9a055536-649b-4840-8443-c7f15c4f961c",
   "metadata": {},
   "source": [
    "### Generative search"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f8bbeb19-3d05-47f4-960f-6f8e9e5dfcc8",
   "metadata": {},
   "source": [
    "A generative search transforms your data at retrieval time. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 51,
   "id": "6eb0d7a5-d31b-4b5a-a7d8-96719947a69e",
   "metadata": {
    "executionCancelledAt": null,
    "executionTime": 4342,
    "lastExecutedAt": 1692032197507,
    "lastScheduledRunId": null,
    "lastSuccessfullyExecutedCode": "response = (\n    client.query\n    .get(\"Question\", [\"question\", \"answer\"])\n    .with_near_text({\"concepts\": [\"history\"]})\n    .with_generate(single_prompt=\"Write a tweet about {question} as an interesting factoid.\")\n    .with_limit(3)\n    .do()\n)\n\njprint(response)",
    "outputsMetadata": {
     "0": {
      "height": 616,
      "type": "stream"
     }
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Based on the movie descriptions, here's an analysis of potential audience demographics:\n",
      "\n",
      "**General Audience:**  All of these movies appeal to a wide range of viewers, but they each have specific audiences within that general group.\n",
      "\n",
      "* **Casual Sci-Fi Fans:** These films all share themes of space exploration and adventure, offering compelling stories with iconic characters that are easily recognizable. \n",
      "    * This audience enjoys escapism through entertainment value and will likely enjoy the comedy elements present in \"Galaxy Quest\". \n",
      "* **Millennial & Gen X Audiences:**  \"Star Wars\", \"Guardians of the Galaxy\", and \"Interstellar\" appeal to a generation raised on these classics, offering nostalgia and emotional connection.\n",
      "* **Family-Friendly Fans:** The themes of heroism and justice, as seen in \"Star Wars\" and \"Guardians of the Galaxy\", are appealing for families with children who enjoy adventure stories. \n",
      "\n",
      "**Specific Demographic Breakdown:**\n",
      "\n",
      "* **\"Galaxy Quest\":**  This film is targeted towards fans of science fiction comedy, especially those who enjoyed classic sci-fi like \"The Hitchhiker's Guide to the Galaxy\" and appreciate meta-humor about the appeal of space movies.\n",
      "* **\"Star Wars\" Trilogy:** This franchise continues to draw in a dedicated audience across generations, from young children who enjoy the adventures of Luke Skywalker and Darth Vader, to adults who revisit familiar characters and storylines with each new film. \n",
      "* **\"Interstellar\":**  The \"interstellar\" space travel adventure appeals to those interested in science fiction that explores complex themes such as time dilation and the vastness of space. This film targets viewers who appreciate visually stunning effects and thought-provoking narratives. \n",
      "* **\"Guardians of the Galaxy Vol. 2\":** This film has a wider appeal due to its unique combination of humor, action, and emotional themes. It appeals to those looking for a lighter sci-fi experience with relatable characters and complex plot twists. \n",
      "* **\"Star Wars: Episode I - The Phantom Menace\":**  This film targets older viewers who enjoyed the original \"Star Wars\" trilogy and are interested in the prequel storyline of Anakin Skywalker's journey.\n",
      "\n",
      "\n",
      "It is important to note that each movie resonates with a certain audience, but many of these movies appeal to multiple demographics depending on their specific themes, plotlines, and storytelling. \n",
      "\n",
      "👽🚀 Did you miss your weekly dose of space adventures? 🌌  \"Galaxy Quest\" is back on my feed and it's *out of this world*! 😂 Tim Allen, Sigourney Weaver & Alan Rickman star as washed-up actors thrust into real-life heroism. You NEED to watch this one! #GalaxyQuest #MustSeeMovie  [link to trailer/movie info] \n",
      "\n",
      "{\n",
      "  \"title\": \"Galaxy Quest\",\n",
      "  \"overview\": \"For four years, the courageous crew of the NSEA protector - \\\"Commander Peter Quincy Taggart\\\" (Tim Allen), \\\"Lt. Tawny Madison (Sigourney Weaver) and \\\"Dr.Lazarus\\\" (Alan Rickman) - set off on a thrilling and often dangerous mission in space...and then their series was cancelled! Now, twenty years later, aliens under attack have mistaken the Galaxy Quest television transmissions for \\\"historical documents\\\" and beam up the crew of has-been actors to save the universe. With no script, no director and no clue, the actors must turn in the performances of their lives.\",\n",
      "  \"year\": 1999,\n",
      "  \"popularity\": 62.01\n",
      "}\n",
      "🚀💥 The Rebel Alliance fights for freedom! 💫  Princess Leia is captured, but Luke, Han & their droid friends are ready to fight for justice. 🔥 Join them in #StarWars – out now! #SpaceOdyssey #ClassicMovieMagic 🍿 \n",
      "\n",
      "**[Link to movie trailer/website]** \n",
      "\n",
      "{\n",
      "  \"title\": \"Star Wars\",\n",
      "  \"overview\": \"Princess Leia is captured and held hostage by the evil Imperial forces in their effort to take over the galactic Empire. Venturesome Luke Skywalker and dashing captain Han Solo team together with the loveable robot duo R2-D2 and C-3PO to rescue the beautiful princess and restore peace and justice in the Empire.\",\n",
      "  \"year\": 1977,\n",
      "  \"popularity\": 250.715\n",
      "}\n",
      "🌍🚀 Witness humanity's greatest leap into the unknown! 🌌  #Interstellar follows a team of explorers venturing through a wormhole for a journey beyond imagination. Prepare to be blown away!  💥 [Link to Movie] #SciFi #SpaceAdventure  🍿✨ \n",
      "\n",
      "{\n",
      "  \"title\": \"Interstellar\",\n",
      "  \"overview\": \"The adventures of a group of explorers who make use of a newly discovered wormhole to surpass the limitations on human space travel and conquer the vast distances involved in an interstellar voyage.\",\n",
      "  \"year\": 2014,\n",
      "  \"popularity\": 283.624\n",
      "}\n",
      "🚀FAMILY & ADVENTURE! #GuardiansOfTheGalaxyVol2 is back and it's even wilder than before.  Peter Quill's past is revealed, enemies rise, and love conquers all! 🌌⚡️  See it NOW! [Link to movie] #Marvel #DisneyPlus \n",
      "\n",
      "**[Add a picture of the cast or poster]**\n",
      "\n",
      "{\n",
      "  \"year\": 2017,\n",
      "  \"title\": \"Guardians of the Galaxy Vol. 2\",\n",
      "  \"overview\": \"The Guardians must fight to keep their newfound family together as they unravel the mysteries of Peter Quill's true parentage.\",\n",
      "  \"popularity\": 142.267\n",
      "}\n",
      "🚀🌌 Witness the dawn of a new saga! 🌌🚀\n",
      "\n",
      "Anakin Skywalker's destiny unfolds in #StarWars: Episode I - The Phantom Menace!  A thrilling adventure where the Force is strong & the evil Sith threaten the galaxy. 🔥💥 #MayThe4thBeWithYou #StarWarsPremiere \n",
      "\n",
      "[Movie poster image] \n",
      "\n",
      "{\n",
      "  \"title\": \"Star Wars: Episode I - The Phantom Menace\",\n",
      "  \"overview\": \"Anakin Skywalker, a young slave strong with the Force, is discovered on Tatooine. Meanwhile, the evil Sith have returned, enacting their plot for revenge against the Jedi.\",\n",
      "  \"year\": 1999,\n",
      "  \"popularity\": 60.19\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "response = movies.generate.near_text(\n",
    "    query=\"galaxy\",\n",
    "    limit=5,\n",
    "    target_vector=\"title\",\n",
    "    single_prompt=\"Write a tweet promoting the movie with TITLE: {title} and OVERVIEW: {overview}.\",\n",
    "    grouped_task=\"What audience demographic might enjoy this group of movies?\"\n",
    ")\n",
    "\n",
    "print(response.generated)\n",
    "for o in response.objects:\n",
    "    print(o.generated)\n",
    "    print(json.dumps(o.properties, indent=2))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "29d4445d-918d-47c1-a553-688fffbb7164",
   "metadata": {},
   "source": [
    "You can see here ⬆️ that each object has been transformed into a tweet by the LLM based on our prompt."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "bdd87179-0c9b-42f8-85f0-8d790dafa4ad",
   "metadata": {},
   "source": [
    "You can ask LLMs to perform all sorts of tasks"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "id": "03472eb3-b993-4c6b-80c5-058d52ca0fbb",
   "metadata": {
    "executionCancelledAt": null,
    "executionTime": 2226,
    "lastExecutedAt": 1692032199733,
    "lastScheduledRunId": null,
    "lastSuccessfullyExecutedCode": "response = (\n    client.query\n    .get(\"Question\", [\"question\", \"answer\"])\n    .with_near_text({\"concepts\": [\"history\"]})\n    .with_generate(single_prompt=\"Translate {question} into French.\")\n    .with_limit(3)\n    .do()\n)\n\njprint(response)",
    "outputsMetadata": {
     "0": {
      "height": 616,
      "type": "stream"
     }
    }
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Après une cancellation, les acteurs d'une série de science-fiction doivent sauver l'univers en interprétant \"Galaxy Quest\". \n",
      "\n",
      "{\n",
      "  \"year\": 1999,\n",
      "  \"title\": \"Galaxy Quest\",\n",
      "  \"overview\": \"For four years, the courageous crew of the NSEA protector - \\\"Commander Peter Quincy Taggart\\\" (Tim Allen), \\\"Lt. Tawny Madison (Sigourney Weaver) and \\\"Dr.Lazarus\\\" (Alan Rickman) - set off on a thrilling and often dangerous mission in space...and then their series was cancelled! Now, twenty years later, aliens under attack have mistaken the Galaxy Quest television transmissions for \\\"historical documents\\\" and beam up the crew of has-been actors to save the universe. With no script, no director and no clue, the actors must turn in the performances of their lives.\",\n",
      "  \"popularity\": 62.01\n",
      "}\n",
      "Leia est en captivité et les forces impérialistes cherchent à instaurer une domination sur l'Empire, alors que Luke, Han et leurs compagnons vont la libérer et restaurer la paix. \n",
      "\n",
      "{\n",
      "  \"year\": 1977,\n",
      "  \"title\": \"Star Wars\",\n",
      "  \"overview\": \"Princess Leia is captured and held hostage by the evil Imperial forces in their effort to take over the galactic Empire. Venturesome Luke Skywalker and dashing captain Han Solo team together with the loveable robot duo R2-D2 and C-3PO to rescue the beautiful princess and restore peace and justice in the Empire.\",\n",
      "  \"popularity\": 250.715\n",
      "}\n",
      "Des explorateurs explorent l'espace grâce à une nouvelle découverte de trou noir et parcourent les immenses distances interstellaires. \n",
      "\n",
      "{\n",
      "  \"title\": \"Interstellar\",\n",
      "  \"overview\": \"The adventures of a group of explorers who make use of a newly discovered wormhole to surpass the limitations on human space travel and conquer the vast distances involved in an interstellar voyage.\",\n",
      "  \"year\": 2014,\n",
      "  \"popularity\": 283.624\n",
      "}\n"
     ]
    }
   ],
   "source": [
    "response = movies.generate.near_text(\n",
    "    query=\"galaxy\",\n",
    "    target_vector=\"title\",\n",
    "    limit=3,\n",
    "    single_prompt=\"Summarise the following movie overview into a short French sentence: {overview}.\"\n",
    ")\n",
    "\n",
    "for o in response.objects:\n",
    "    print(o.generated)\n",
    "    print(json.dumps(o.properties, indent=2))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0e03f4fb-8a08-41a7-a7d1-dc1ebc14fae3",
   "metadata": {},
   "outputs": [],
   "source": [
    "client.close()"
   ]
  }
 ],
 "metadata": {
  "editor": "DataCamp Workspace",
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.11.9"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}