basics/template_query.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "f72bea19-a48c-4a6c-96e5-1e5c98646be2",
   "metadata": {},
   "source": [
    "# Template Query"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "aaf7b111-39a2-4a8e-8d6d-0c8079322feb",
   "metadata": {},
   "source": [
    "This notebook demostrates the use of [template query](https://docs.tigergraph.com/graph-ml/current/using-an-algorithm/#_packaged_template_queries), which is a new feature since TigerGraph Database `3.9` and pyTigerGraph `1.3`. That means, this notebook only runs with DB 3.9 and above and pyTigerGraph 1.3 and above.\n",
    "\n",
    "## What are template queries?\n",
    "\n",
    "Template queries, in this context, are the \"static\" version of the [graph algorithms](https://docs.tigergraph.com/graph-ml/current/intro/). \"Static\" means that a query is bound to the vertex type(s) and/or edge type(s) given to a query as input parameters at installation time. If you change the input vertex or edge types later, a new query will be generated and installed. \n",
    "\n",
    "But note not every graph algorithm has a template query currently. More template queries will be added in future versions.\n",
    "\n",
    "## How is current user experience impacted?\n",
    "\n",
    "As a user, there is not much difference in calling a template graph algorithm (See below for examples). You will only notice the query installation when you change input vertex or edge types. Changing other query parameters such as `iterations` won't generate a new query. \n",
    "\n",
    "## What is the benefit of using template queries?\n",
    "\n",
    "As a template query is bound to certain vertex and edge types, it runs  faster than the \"schema-less\" version. Therefore, it is useful when speed is the main concern. However, there is a tradeoff of flexibility when you are experimenting with vertex and edge types.  \n",
    "\n",
    "## Examples"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "c1635df7-998a-4649-aba2-6ee00a973d12",
   "metadata": {},
   "source": [
    "### Connection to Database\n",
    "\n",
    "The `TigerGraphConnection` class represents a connection to the TigerGraph database. Under the hood, it stores the necessary information to communicate with the database. It is able to perform quite a few database tasks. Please see its [documentation](https://docs.tigergraph.com/pytigergraph/current/intro/) for details.\n",
    "\n",
    "To connect your database, modify the `config.json` file accompanying this notebook. Set the value of `getToken` based on whether token auth is enabled for your database. Token auth is always enabled for tgcloud databases. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "8b5dd915-2645-4e4d-ae16-33ed63c1a02d",
   "metadata": {},
   "outputs": [],
   "source": [
    "from pyTigerGraph import TigerGraphConnection\n",
    "import json\n",
    "\n",
    "# Read in DB configs\n",
    "with open('../config.json', \"r\") as config_file:\n",
    "    config = json.load(config_file)\n",
    "    \n",
    "conn = TigerGraphConnection(\n",
    "    host=config[\"host\"],\n",
    "    username=config[\"username\"],\n",
    "    password=config[\"password\"]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "84febfb9-ff4d-4d46-8a45-f8ad6e59c7ce",
   "metadata": {},
   "source": [
    "### Ingest Data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "ddd2fa65-40a0-44b2-9335-3d109de1239f",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "A folder with name ldbc_snb already exists in ./tmp. Skip downloading.\n"
     ]
    }
   ],
   "source": [
    "from pyTigerGraph.datasets import Datasets\n",
    "\n",
    "dataset = Datasets(\"ldbc_snb\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "10063fbb-5522-40cc-82c4-c33ae0a5f3d3",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "---- Checking database ----\n",
      "A graph with name ldbc_snb already exists in the database. Skip ingestion.\n",
      "Graph name is set to ldbc_snb for this connection.\n"
     ]
    }
   ],
   "source": [
    "conn.ingestDataset(dataset, getToken=config[\"getToken\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "cdac1aa7-f786-42f2-9d28-d685bc3c4cb5",
   "metadata": {},
   "source": [
    "### Visualize Schema"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "583574f7-3bf9-4869-b88e-84c237f2ddd2",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "d0c3b6df24c4438081e4b43557f2aade",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "CytoscapeWidget(cytoscape_layout={'name': 'circle', 'animate': True, 'padding': 1}, cytoscape_style=[{'selecto…"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from pyTigerGraph.visualization import drawSchema\n",
    "\n",
    "drawSchema(conn.getSchema(force=True))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0827e34f-4c8f-4bed-8da0-3912303eac72",
   "metadata": {},
   "source": [
    "### Featurizer\n",
    "\n",
    "`pyTigerGraph` provides the `featurizer` as a friendly interface to the graph algorithms. Please see the `feature_engineering` notebook for details on the `featurizer` and the notebooks under `algos` folder for details on the algorithms. Below we briefy review how to run a non-template graph algorithm with the featurizer first, and then we will learn how to run the template version with just one change of the parameters."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b00977f6-bce6-465a-b570-b475d0975924",
   "metadata": {
    "tags": []
   },
   "source": [
    "### Example 1: PageRank\n",
    "\n",
    "#### Non-Template Query "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "64ce67bf-e68a-4174-b653-45b21e8da468",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cannot read manifest file. Trying master branch.\n"
     ]
    }
   ],
   "source": [
    "# Create a featurizer\n",
    "f = conn.gds.featurizer()\n",
    "\n",
    "# Run an algorithm with paramters\n",
    "params = {\n",
    "    'v_type': 'Person', \n",
    "    'e_type': 'Knows', \n",
    "    'max_change': 0.001, \n",
    "    'maximum_iteration': 25, \n",
    "    'damping': 0.85,\n",
    "    'top_k': 10, \n",
    "    'print_results': True, \n",
    "    'result_attribute': '', \n",
    "    'file_path': '', \n",
    "    'display_edges': False}\n",
    "\n",
    "res = f.runAlgorithm(\n",
    "    'tg_pagerank', \n",
    "    params=params\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "11787b46-8ad0-40ba-a1ad-008d7bbd0039",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'@@top_scores_heap': [{'Vertex_ID': '2199023262543', 'score': 24.85992},\n",
       "   {'Vertex_ID': '6597069777240', 'score': 23.86707},\n",
       "   {'Vertex_ID': '17592186053137', 'score': 23.6497},\n",
       "   {'Vertex_ID': '4398046513018', 'score': 23.56558},\n",
       "   {'Vertex_ID': '30786325585162', 'score': 23.43321},\n",
       "   {'Vertex_ID': '2199023259756', 'score': 22.87003},\n",
       "   {'Vertex_ID': '24189255819727', 'score': 22.31711},\n",
       "   {'Vertex_ID': '19791209302403', 'score': 20.59326},\n",
       "   {'Vertex_ID': '8796093029267', 'score': 20.49563},\n",
       "   {'Vertex_ID': '4139', 'score': 20.41319}]}]"
      ]
     },
     "execution_count": 6,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check result\n",
    "res"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "id": "eae59324-1e47-4f08-9e0a-b2160c37008a",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time elapsed: 1.36 seconds\n"
     ]
    }
   ],
   "source": [
    "#Rerun the algorithm and record its run time for comparison later\n",
    "import time\n",
    "\n",
    "start_time = time.perf_counter()\n",
    "res = f.runAlgorithm(\n",
    "    'tg_pagerank', \n",
    "    params=params\n",
    ")\n",
    "non_template_time = time.perf_counter() - start_time\n",
    "print(\"Time elapsed: {:.3} seconds\".format(non_template_time))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ea476c4e-50f8-4f12-b62b-7e1ec72b3cb0",
   "metadata": {},
   "source": [
    "#### Template Query\n",
    "\n",
    "To use template query, there is only one change: set `templateQuery` to `True` when running an algorithm with the featurizer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "e7629332-95f2-41b1-92f0-be532e93eba2",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cannot read manifest file. Trying master branch.\n"
     ]
    }
   ],
   "source": [
    "# Create a featurizer\n",
    "f = conn.gds.featurizer()\n",
    "\n",
    "# Run an algorithm with paramters\n",
    "params = {\n",
    "    'v_type': 'Person', \n",
    "    'e_type': 'Knows', \n",
    "    'max_change': 0.001, \n",
    "    'maximum_iteration': 25, \n",
    "    'damping': 0.85,\n",
    "    'top_k': 10, \n",
    "    'print_results': True, \n",
    "    'result_attribute': '', \n",
    "    'file_path': '', \n",
    "    'display_edges': False}\n",
    "\n",
    "res = f.runAlgorithm(\n",
    "    'tg_pagerank', \n",
    "    params=params,\n",
    "    templateQuery=True # Set this to True to use template query. Default False.\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "a3fb8010-e208-419f-895b-cc9d9cb14f00",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'@@top_scores_heap': [{'score': 24.85992, 'Vertex_ID': '2199023262543'},\n",
       "   {'score': 23.86707, 'Vertex_ID': '6597069777240'},\n",
       "   {'score': 23.6497, 'Vertex_ID': '17592186053137'},\n",
       "   {'score': 23.56558, 'Vertex_ID': '4398046513018'},\n",
       "   {'score': 23.4332, 'Vertex_ID': '30786325585162'},\n",
       "   {'score': 22.87003, 'Vertex_ID': '2199023259756'},\n",
       "   {'score': 22.3171, 'Vertex_ID': '24189255819727'},\n",
       "   {'score': 20.59327, 'Vertex_ID': '19791209302403'},\n",
       "   {'score': 20.49563, 'Vertex_ID': '8796093029267'},\n",
       "   {'score': 20.41318, 'Vertex_ID': '4139'}]}]"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check result\n",
    "res"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "ffe060bf-9df7-4431-a7d2-b5f9235f766e",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time elapsed: 0.708 seconds\n"
     ]
    }
   ],
   "source": [
    "# Rerun the template query and record its run time.\n",
    "\n",
    "start_time = time.perf_counter()\n",
    "res = f.runAlgorithm(\n",
    "    'tg_pagerank', \n",
    "    params=params,\n",
    "    templateQuery=True\n",
    ")\n",
    "template_time = time.perf_counter() - start_time\n",
    "print(\"Time elapsed: {:.3} seconds\".format(template_time))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "e0a18b87-c639-446b-8400-f43be6a966da",
   "metadata": {},
   "source": [
    "### Example 2: Breadth-First Search\n",
    "\n",
    "#### Non-Template Query "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "id": "53541f17-f179-401c-92ad-31a9f324f5f9",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cannot read manifest file. Trying master branch.\n"
     ]
    }
   ],
   "source": [
    "# Create a featurizer\n",
    "f = conn.gds.featurizer()\n",
    "\n",
    "# Run an algorithm with paramters\n",
    "params = {\n",
    "    \"v_type_set\": [\"Person\"],\n",
    "    \"e_type_set\": [\"Knows\"],\n",
    "    \"max_hops\": 2,\n",
    "    \"v_start\": {\"id\": \"21990232556463\", \"type\": \"Person\"}, ##{\"id\": \"vertex_id\", \"type\": \"vertex_type\"}\n",
    "    \"print_results\": True,\n",
    "    \"result_attribute\": \"\",\n",
    "    \"file_path\": \"\",\n",
    "    \"display_edges\": False\n",
    "}\n",
    "\n",
    "res = f.runAlgorithm(\n",
    "    'tg_bfs', \n",
    "    params=params\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "id": "13f1e195-8413-4144-894c-9483995e3929",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'v_id': '30786325580605',\n",
       "  'v_type': 'Person',\n",
       "  'attributes': {'Start.@sum_step': 2}},\n",
       " {'v_id': '13194139540951',\n",
       "  'v_type': 'Person',\n",
       "  'attributes': {'Start.@sum_step': 2}},\n",
       " {'v_id': '6597069769055',\n",
       "  'v_type': 'Person',\n",
       "  'attributes': {'Start.@sum_step': 2}},\n",
       " {'v_id': '15393162796423',\n",
       "  'v_type': 'Person',\n",
       "  'attributes': {'Start.@sum_step': 2}},\n",
       " {'v_id': '15393162792715',\n",
       "  'v_type': 'Person',\n",
       "  'attributes': {'Start.@sum_step': 2}},\n",
       " {'v_id': '28587302332123',\n",
       "  'v_type': 'Person',\n",
       "  'attributes': {'Start.@sum_step': 2}},\n",
       " {'v_id': '6597069774914',\n",
       "  'v_type': 'Person',\n",
       "  'attributes': {'Start.@sum_step': 2}},\n",
       " {'v_id': '13194139542969',\n",
       "  'v_type': 'Person',\n",
       "  'attributes': {'Start.@sum_step': 2}},\n",
       " {'v_id': '15393162795179',\n",
       "  'v_type': 'Person',\n",
       "  'attributes': {'Start.@sum_step': 2}},\n",
       " {'v_id': '4398046519923',\n",
       "  'v_type': 'Person',\n",
       "  'attributes': {'Start.@sum_step': 2}}]"
      ]
     },
     "execution_count": 16,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check result\n",
    "res[0]['Start'][:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "id": "743fa651-2d95-4ada-b67a-2268596e8ee8",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Time elapsed: 0.14 seconds\n"
     ]
    }
   ],
   "source": [
    "#Rerun the algorithm and record its run time for comparison later\n",
    "import time\n",
    "\n",
    "start_time = time.perf_counter()\n",
    "res = f.runAlgorithm(\n",
    "    'tg_bfs', \n",
    "    params=params\n",
    ")\n",
    "bfs_non_template_time = time.perf_counter() - start_time\n",
    "print(\"Time elapsed: {:.3} seconds\".format(bfs_non_template_time))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b5fdc85e-b41c-4ffe-bd51-7231f54b7e26",
   "metadata": {},
   "source": [
    "#### Template Query\n",
    "\n",
    "To use template query, there is only one change: set `templateQuery` to `True` when running an algorithm with the featurizer."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "id": "3c2c7e9f-212f-492e-8063-03b5904ed703",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Cannot read manifest file. Trying master branch.\n",
      "Running the algorithm. It might take a minute to install the query if this is the first time it runs.\n"
     ]
    }
   ],
   "source": [
    "# Create a featurizer\n",
    "f = conn.gds.featurizer()\n",
    "\n",
    "# Run an algorithm with paramters\n",
    "params = {\n",
    "    \"v_type_set\": [\"Person\"],\n",
    "    \"e_type_set\": [\"Knows\"],\n",
    "    \"max_hops\": 2,\n",
    "    \"v_start\": {\"id\": \"21990232556463\", \"type\": \"Person\"}, ##{\"id\": \"vertex_id\", \"type\": \"vertex_type\"}\n",
    "    \"print_results\": True,\n",
    "    \"result_attribute\": \"\",\n",
    "    \"file_path\": \"\",\n",
    "    \"display_edges\": False\n",
    "}\n",
    "\n",
    "res = f.runAlgorithm(\n",
    "    'tg_bfs', \n",
    "    params=params,\n",
    "    templateQuery=True # Set this to True to use template query. Default False.\n",
    ")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "id": "fb976464-906b-4f92-8fba-dcad76e94289",
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "[{'v_id': '30786325580605',\n",
       "  'attributes': {'Start.@sum_step': 2},\n",
       "  'v_type': 'Person'},\n",
       " {'v_id': '13194139540951',\n",
       "  'attributes': {'Start.@sum_step': 2},\n",
       "  'v_type': 'Person'},\n",
       " {'v_id': '6597069769055',\n",
       "  'attributes': {'Start.@sum_step': 2},\n",
       "  'v_type': 'Person'},\n",
       " {'v_id': '15393162796423',\n",
       "  'attributes': {'Start.@sum_step': 2},\n",
       "  'v_type': 'Person'},\n",
       " {'v_id': '15393162792715',\n",
       "  'attributes': {'Start.@sum_step': 2},\n",
       "  'v_type': 'Person'},\n",
       " {'v_id': '28587302332123',\n",
       "  'attributes': {'Start.@sum_step': 2},\n",
       "  'v_type': 'Person'},\n",
       " {'v_id': '6597069774914',\n",
       "  'attributes': {'Start.@sum_step': 2},\n",
       "  'v_type': 'Person'},\n",
       " {'v_id': '9079', 'attributes': {'Start.@sum_step': 2}, 'v_type': 'Person'},\n",
       " {'v_id': '21990232561273',\n",
       "  'attributes': {'Start.@sum_step': 2},\n",
       "  'v_type': 'Person'},\n",
       " {'v_id': '15393162792433',\n",
       "  'attributes': {'Start.@sum_step': 2},\n",
       "  'v_type': 'Person'}]"
      ]
     },
     "execution_count": 19,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "# Check result\n",
    "res[0]['Start'][:10]"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "cccc25d0-4074-4e87-a1ba-6cb44e682af4",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Running the algorithm. It might take a minute to install the query if this is the first time it runs.\n",
      "Time elapsed: 0.146 seconds\n"
     ]
    }
   ],
   "source": [
    "# Rerun the template query and record its run time.\n",
    "\n",
    "start_time = time.perf_counter()\n",
    "res = f.runAlgorithm(\n",
    "    'tg_bfs', \n",
    "    params=params,\n",
    "    templateQuery=True\n",
    ")\n",
    "bfs_template_time = time.perf_counter() - start_time\n",
    "print(\"Time elapsed: {:.3} seconds\".format(bfs_template_time))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ec026b5a-b632-47a9-b89d-64ade8d33eb4",
   "metadata": {},
   "source": [
    "### Takeaways"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 25,
   "id": "c77069ac-1e53-42bc-aaea-0ed865baef90",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The template version of PageRank is 47% faster than the non-template version.\n"
     ]
    }
   ],
   "source": [
    "print(\n",
    "    \"The template version of PageRank is {}% faster than the non-template version.\".format(\n",
    "    int(100*(non_template_time-template_time)/non_template_time)))\n",
    "\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 29,
   "id": "47fa15eb-dd6f-4da3-b548-6dc645f3cb3c",
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "The template and non-template versions of BFS show almost the same performance (0.14555528794880956 v.s. 0.14016598195303231) as this graph is small.\n"
     ]
    }
   ],
   "source": [
    "print(\n",
    "    \"The template and non-template versions of BFS show almost the same performance ({} v.s. {}) as this graph is small.\".format(\n",
    "    bfs_template_time, bfs_non_template_time))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0c358c8f-ab85-49fb-a7c3-504a526392ac",
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "PyTorch",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.13"
  },
  "vscode": {
   "interpreter": {
    "hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
   }
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}