quickstarts/python_quickstart.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "bb2094b8-13a5-4f7c-bd21-d2c709dab914",
   "metadata": {},
   "source": [
    "# Quickstart: Hello, KDB.AI\n",
    "\n",
    "How to get started with the KDB.AI vector database. Here, you'll get a quick taste of KDB.AI in  ~10 minutes.\n",
    "\n",
    "You will learn how to:\n",
    "\n",
    "1. Connect to KDB.AI\n",
    "1. Create a KDB.AI Table\n",
    "1. Add Data to the KDB.AI Table\n",
    "1. Query the Table\n",
    "1. Perform Similarity Search\n",
    "1. Delete the KDB.AI Table"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "260d0f4b-ef09-4bd2-a197-a9351be24684",
   "metadata": {},
   "source": [
    "## 0. Setup"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d1468bd3",
   "metadata": {},
   "source": [
    "### Install dependencies \n",
    "\n",
    "In order to successfully run this sample, the [Setup](https://github.com/KxSystems/kdbai-samples/blob/main/README.md#setup) steps in the repository's `README.md` file must be completed.\n",
    "This will ensure that you have installed all of the relevant packages and versions needed for this sample.\n",
    "If you have not completed these setup steps, please navigate to the repositories `README.md` file and follow the steps detailed there."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cc6d17b7",
   "metadata": {},
   "source": [
    "### Import Packages"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "805d97da",
   "metadata": {},
   "outputs": [],
   "source": [
    "# vector DB\n",
    "import os\n",
    "from getpass import getpass\n",
    "import kdbai_client as kdbai\n",
    "import time"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "a55ae34e-472b-4aa7-9add-1fcb2ee24a41",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy as np\n",
    "import pandas as pd"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8c660c7d",
   "metadata": {},
   "source": [
    "## 1. Connect to KDB.AI"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d3a3aa22",
   "metadata": {},
   "source": [
    "### Define KDB.AI Session\n",
    "\n",
    "KDB.AI comes in two offerings:\n",
    "\n",
    "1. [KDB.AI Cloud](https://trykdb.kx.com/kdbai/signup/) - For experimenting with smaller generative AI projects with a vector database in our cloud.\n",
    "2. [KDB.AI Server](https://trykdb.kx.com/kdbaiserver/signup/) - For evaluating large scale generative AI applications on-premises or on your own cloud provider.\n",
    "\n",
    "Depending on which you use there will be different setup steps and connection details required.\n",
    "\n",
    "##### Option 1. KDB.AI Cloud\n",
    "\n",
    "To use KDB.AI Cloud, you will need two session details - a URL endpoint and an API key.\n",
    "To get these you can sign up for free [here](https://trykdb.kx.com/kdbai/signup).\n",
    "\n",
    "You can connect to a KDB.AI Cloud session using `kdbai.Session` and passing the session URL endpoint and API key details from your KDB.AI Cloud portal.\n",
    "\n",
    "If the environment variables `KDBAI_ENDPOINTS` and `KDBAI_API_KEY` exist on your system containing your KDB.AI Cloud portal details, these variables will automatically be used to connect.\n",
    "If these do not exist, it will prompt you to enter your KDB.AI Cloud portal session URL endpoint and API key details."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "2e85c1ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "KDBAI_ENDPOINT = (\n",
    "    os.environ[\"KDBAI_ENDPOINT\"]\n",
    "    if \"KDBAI_ENDPOINT\" in os.environ\n",
    "    else input(\"KDB.AI endpoint: \")\n",
    ")\n",
    "KDBAI_API_KEY = (\n",
    "    os.environ[\"KDBAI_API_KEY\"]\n",
    "    if \"KDBAI_API_KEY\" in os.environ\n",
    "    else getpass(\"KDB.AI API key: \")\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "6f330098",
   "metadata": {},
   "outputs": [],
   "source": [
    "session = kdbai.Session(api_key=KDBAI_API_KEY, endpoint=KDBAI_ENDPOINT)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "86f16226",
   "metadata": {},
   "source": [
    "##### Option 2. KDB.AI Server\n",
    "\n",
    "To use KDB.AI Server, you will need download and run your own container.\n",
    "To do this, you will first need to sign up for free [here](https://trykdb.kx.com/kdbaiserver/signup/). \n",
    "\n",
    "You will receive an email with the required license file and bearer token needed to download your instance.\n",
    "Follow instructions in the signup email to get your session up and running.\n",
    "\n",
    "Once the [setup steps](https://code.kx.com/kdbai/gettingStarted/kdb-ai-server-setup.html) are complete you can then connect to your KDB.AI Server session using `kdbai.Session` and passing your local endpoint."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ff249f70",
   "metadata": {},
   "outputs": [],
   "source": [
    "# session = kdbai.Session(endpoint=\"http://localhost:8082\")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1ec2c77b",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-info\">\n",
    "<b>Need help understanding a function?</b><br/>\n",
    "Add ? before or after any function name in KDB.AI to bring up the documentation for that function along with sample code and arguments.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "6e54917b",
   "metadata": {},
   "outputs": [],
   "source": [
    "?kdbai.Session"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8788a6b1",
   "metadata": {},
   "source": [
    "### Verify Defined Tables\n",
    "\n",
    "We can check our connection using the `session.list()` function.\n",
    "This will return a list of all the tables we have defined in our vector database thus far.\n",
    "This should return an empty list."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "97e5f4a9",
   "metadata": {},
   "outputs": [],
   "source": [
    "# ensure no table called \"data\" exists\n",
    "try:\n",
    "    session.table(\"data\").drop()\n",
    "    time.sleep(5)\n",
    "except kdbai.KDBAIException:\n",
    "    pass"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "7877f51c",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[]"
      ]
     },
     "execution_count": 7,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "session.list()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e33f03c3",
   "metadata": {},
   "source": [
    "## 2. Create a KDB.AI Table\n",
    "\n",
    "To create a table we can use `create_table`, this function takes two arguments - the name and schema of the table.\n",
    "\n",
    "This schema must meet the following criteria:\n",
    "- It must contain a list of columns. \n",
    "- All columns must have either a `pytype` or a `qtype` specified, except the column of vectors.\n",
    "- One column of vector embeddings may also have a `vectorIndex` attribute with the configuration of the index for similarity search - this column is implicitly an array of `float32`.\n",
    "\n",
    "Run `?session.create_table` for more details and sample code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "6f40400b",
   "metadata": {},
   "outputs": [],
   "source": [
    "?session.create_table"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9da55253",
   "metadata": {},
   "source": [
    "### Define Schema \n",
    "\n",
    "Our table will have two columns the first `id` with a list of dummy ID's, the second will be the vector embeddings we will use for similarity search later on in this example. \n",
    "\n",
    "We will define our dimensionality, similarity metric and index type with the `vectorIndex` attribute. For this example we chose:\n",
    "- `dims = 8` : In the next section, we generate embeddings that are eight-dimensional to match this. You can chose any value here.\n",
    "- `metric = L2` : We chose [L2/Euclidean distance](https://en.wikipedia.org/wiki/Euclidean_distance). Our dummy dataset is low dimensional which Euclidean distance is suitable for. You have the choice of using other metrics here like [IP/Inner Product](https://en.wikipedia.org/wiki/Inner_product_space) and [CS/Cosine Similarity](https://en.wikipedia.org/wiki/Cosine_similarity) and the one you chose depends on the specific context and nature of your data.\n",
    "- `type = flat` : We use a [Flat index](https://faiss.ai/cpp_api/struct/structfaiss_1_1IndexFlat.html) here as we have a simple data structure so this is more than adequate. You have the choice of using other indexes like [HNSW](https://faiss.ai/cpp_api/struct/structfaiss_1_1IndexHNSW.html) and [IVFPQ](https://faiss.ai/cpp_api/struct/structfaiss_1_1IndexIVFPQ.html) here, as with metrics the one you chose depends your data and your overall performance requirements.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e5e8b782",
   "metadata": {},
   "outputs": [],
   "source": [
    "schema = {\n",
    "    \"columns\": [\n",
    "        {\"name\": \"id\", \"pytype\": \"str\"},\n",
    "        {\"name\": \"vectors\", \"vectorIndex\": {\"dims\": 8, \"metric\": \"L2\", \"type\": \"flat\"}},\n",
    "    ]\n",
    "}"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09a5caa0",
   "metadata": {},
   "source": [
    "### Create Table"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "34067680",
   "metadata": {},
   "outputs": [],
   "source": [
    "table = session.create_table(\"data\", schema)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "20afbea1",
   "metadata": {},
   "source": [
    "## 3. Add Data to the KDB.AI Table\n",
    "\n",
    "First, generate a vector of five 8-dimensional vectors which will be the vector embeddings in this example. We will then add these to pandas dataframe with column names/types matching the target table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "37581e86",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Create a NumPy array of 5 eight-dimensional float32 arrays\n",
    "vectors = np.array(\n",
    "    [\n",
    "        [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8],\n",
    "        [0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9],\n",
    "        [0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0],\n",
    "        [0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1],\n",
    "        [0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2],\n",
    "    ],\n",
    "    dtype=np.float32,\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "f5dc41e8",
   "metadata": {},
   "outputs": [],
   "source": [
    "# Example ID values\n",
    "ids = [\"h\", \"e\", \"l\", \"l\", \"o\"]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "730c9f08",
   "metadata": {},
   "outputs": [],
   "source": [
    "# column names/types matching the schema\n",
    "embeddings = pd.DataFrame({\"id\": ids, \"vectors\": list(vectors)})"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "4a31f878",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>vectors</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>h</td>\n",
       "      <td>[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>e</td>\n",
       "      <td>[0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>l</td>\n",
       "      <td>[0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>l</td>\n",
       "      <td>[0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>o</td>\n",
       "      <td>[0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  id                                   vectors\n",
       "0  h  [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]\n",
       "1  e  [0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]\n",
       "2  l  [0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]\n",
       "3  l  [0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1]\n",
       "4  o  [0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2]"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "embeddings"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "43cd2ad8",
   "metadata": {},
   "source": [
    "We can now add data to our KDB.AI table using `insert`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b7e0f8c5",
   "metadata": {},
   "outputs": [],
   "source": [
    "table.insert(embeddings)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "09577e8e",
   "metadata": {},
   "source": [
    "## 4. Query the Table\n",
    "\n",
    "We can use `query` to query data from the table."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "f4b8b8e5",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>vectors</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>h</td>\n",
       "      <td>[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>e</td>\n",
       "      <td>[0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>l</td>\n",
       "      <td>[0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>3</th>\n",
       "      <td>l</td>\n",
       "      <td>[0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>4</th>\n",
       "      <td>o</td>\n",
       "      <td>[0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  id                                   vectors\n",
       "0  h  [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]\n",
       "1  e  [0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]\n",
       "2  l  [0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]\n",
       "3  l  [0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1]\n",
       "4  o  [0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1, 0.2]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "table.query()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "55e69a5b",
   "metadata": {},
   "source": [
    "The `query` function accepts a wide range of arguments to make it easy to filter, aggregate, and sort.\n",
    "Run `?table.query` to see them all.\n",
    "\n",
    "Let's filter for ids equal to 'l' only. Two rows are returned as expected."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "id": "41c6f156",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>vectors</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>l</td>\n",
       "      <td>[0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>l</td>\n",
       "      <td>[0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1]</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  id                                   vectors\n",
       "0  l  [0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]\n",
       "1  l  [0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1]"
      ]
     },
     "execution_count": 17,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "table.query(filter=[(\"like\", \"id\", \"l\")])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9c267a58",
   "metadata": {},
   "source": [
    "## 5. Perform Similarity Search\n",
    "\n",
    "Finally, let's perform similarity search on the table. We do this using the `search` function."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "595829ff",
   "metadata": {},
   "outputs": [],
   "source": [
    "?table.search"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9bb341f3",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-warning\">\n",
    "<b>Note:</b> The dimension of input query vectors must match the vector embedding dimensions in the table, defined in schema above.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "c9301c97",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[  id                                   vectors  __nn_distance\n",
       " 0  e  [0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]           0.01]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the closest neighbor of a single query vector\n",
    "table.search(vectors=[[0.1, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]], n=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "49758e9d",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-warning\">\n",
    "<b>Note:</b> The output was a list of length one, matching the number of vectors we input to the search. This can be indexed on position [0] to extract the dataframe corresponding to the single input vector.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f0bc3820",
   "metadata": {},
   "source": [
    "The closest matching neighbor for the query vector passed in is returned alongside the calculation of L2 ([Euclidean Distance](#https://en.wikipedia.org/wiki/Euclidean_distance)) similarity.\n",
    "\n",
    "We can also rerun the same query for more neighbors."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "id": "0c0cdda8",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>vectors</th>\n",
       "      <th>__nn_distance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>e</td>\n",
       "      <td>[0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]</td>\n",
       "      <td>0.01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>h</td>\n",
       "      <td>[0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]</td>\n",
       "      <td>0.07</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td>l</td>\n",
       "      <td>[0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]</td>\n",
       "      <td>0.11</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  id                                   vectors  __nn_distance\n",
       "0  e  [0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]           0.01\n",
       "1  h  [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]           0.07\n",
       "2  l  [0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]           0.11"
      ]
     },
     "execution_count": 20,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find 3 closest neighbours of a single query vector\n",
    "table.search(vectors=[[0.1, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]], n=3)[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0c9b0e8a",
   "metadata": {},
   "source": [
    "And we can apply a filter to the search results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "id": "f1914948",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/html": [
       "<div>\n",
       "<style scoped>\n",
       "    .dataframe tbody tr th:only-of-type {\n",
       "        vertical-align: middle;\n",
       "    }\n",
       "\n",
       "    .dataframe tbody tr th {\n",
       "        vertical-align: top;\n",
       "    }\n",
       "\n",
       "    .dataframe thead th {\n",
       "        text-align: right;\n",
       "    }\n",
       "</style>\n",
       "<table border=\"1\" class=\"dataframe\">\n",
       "  <thead>\n",
       "    <tr style=\"text-align: right;\">\n",
       "      <th></th>\n",
       "      <th>id</th>\n",
       "      <th>vectors</th>\n",
       "      <th>__nn_distance</th>\n",
       "    </tr>\n",
       "  </thead>\n",
       "  <tbody>\n",
       "    <tr>\n",
       "      <th>0</th>\n",
       "      <td>l</td>\n",
       "      <td>[0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]</td>\n",
       "      <td>1.100000e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>1</th>\n",
       "      <td>l</td>\n",
       "      <td>[0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1]</td>\n",
       "      <td>9.699999e-01</td>\n",
       "    </tr>\n",
       "    <tr>\n",
       "      <th>2</th>\n",
       "      <td></td>\n",
       "      <td>[]</td>\n",
       "      <td>3.402823e+38</td>\n",
       "    </tr>\n",
       "  </tbody>\n",
       "</table>\n",
       "</div>"
      ],
      "text/plain": [
       "  id                                   vectors  __nn_distance\n",
       "0  l  [0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]   1.100000e-01\n",
       "1  l  [0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0, 0.1]   9.699999e-01\n",
       "2                                           []   3.402823e+38"
      ]
     },
     "execution_count": 21,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find 3 closest neighbours of a single query vector\n",
    "table.search(\n",
    "    vectors=[[0.1, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]],\n",
    "    n=3,\n",
    "    filter=[(\"like\", \"id\", \"l\")],\n",
    ")[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "02c1b38d",
   "metadata": {},
   "source": [
    "And also we can search passing more than one query vector."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "id": "c6964035",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[  id                                   vectors  __nn_distance\n",
       " 0  e  [0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]           0.01\n",
       " 1  h  [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]           0.07\n",
       " 2  l  [0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]           0.11,\n",
       "   id                                   vectors  __nn_distance\n",
       " 0  h  [0.1, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8]           0.04\n",
       " 1  e  [0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9]           0.08\n",
       " 2  l  [0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9, 1.0]           0.28]"
      ]
     },
     "execution_count": 22,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Find the 3 closest neighbours of 2 query vectors\n",
    "table.search(\n",
    "    vectors=[\n",
    "        [0.1, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8, 0.9],\n",
    "        [0.3, 0.2, 0.3, 0.4, 0.5, 0.6, 0.7, 0.8],\n",
    "    ],\n",
    "    n=3,\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d8aed9bc-72b2-4e70-b763-e7ce054557db",
   "metadata": {},
   "source": [
    "## 6. Delete the KDB.AI Table\n",
    "\n",
    "We can use `table.drop()` to delete a table. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "548a9d95-aac3-4d63-a87a-99eedfe55f07",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 23,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "table.drop()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "8bc6d801-1371-48d0-98b4-0baa53bc8446",
   "metadata": {},
   "source": [
    "<div class=\"alert alert-block alert-warning\">\n",
    "<b>Warning:</b> Once you drop a table, you cannot use it again.\n",
    "</div>"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "a7672241-42b0-4798-90d7-95aa9fefe68c",
   "metadata": {},
   "source": [
    "## Next Steps\n",
    "\n",
    "Now that you’re successfully making indexes with KDB.AI, you can start inserting your own data or view more examples:\n",
    "- [PDF Document Search](../document_search)\n",
    "- [MRI Image Search](../image_search)\n",
    "- [Music Recommendation System](../music_recommendation)\n",
    "- [Sensor Pattern Matching](../pattern_matching)\n",
    "- [Retrieval Augmented Generation with LangChain](../retrieval_augmented_generation)\n",
    "- [Sentiment Analysis of Reviews](../sentiment_analysis)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.16"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}