Merge pull request #548 from Mats-SX/notebook-updates

Prepare notebook for field testing

Author: FlorentinD

examples/dev/aura-only-features.ipynb

@@ -27,6 +27,19 @@
     "In this notebook, we will illustrate just briefly what example data we are using."
    ]
   },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "# Just to begin, let's make sure we have the correct version of the GDS Python Client installed\n",
+    "\n",
+    "from graphdatascience import __version__\n",
+    "\n",
+    "assert __version__ == \"1.9a1\""
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -40,7 +53,7 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "db_id = \"YOU_DATABASE_ID\"\n",
+    "db_id = \"YOUR_DATABASE_ID\"\n",
     "db_password = \"YOUR_DATABASE_PASSWORD\""
    ]
   },
@@ -59,17 +72,17 @@
    "source": [
     "import os\n",
     "\n",
-    "# AuraDB data ingestion\n",
     "from neo4j import GraphDatabase\n",
     "from graphdatascience.query_runner.aura_db_arrow_query_runner import AuraDbConnectionInfo\n",
     "\n",
-    "# We need to tell the GDS client that we are working with a devenvironment. This does not need to be set in production.\n",
+    "# We need to tell the GDS client that we are working with a devenvironment.\n",
+    "# This does not need to be set in production.\n",
     "os.environ[\"AURA_ENV\"] = \"devstrawberryfield\"\n",
     "\n",
     "db_connection_info = AuraDbConnectionInfo(\n",
     "    f\"neo4j+s://{db_id}-{os.environ['AURA_ENV']}.databases.neo4j-dev.io\", (\"neo4j\", db_password)\n",
     ")\n",
-    "# start a Driver\n",
+    "# start a standard Neo4j Python Driver to connect to the AuraDB instance\n",
     "driver = GraphDatabase.driver(db_connection_info.uri, auth=db_connection_info.auth)\n",
     "\n",
     "# try out our connection\n",
@@ -198,13 +211,13 @@
     "The GDS session offers all the GDS functionality that we are familiar with from AuraDS.\n",
     "However, since the idea is to offload database work to AuraDB, the GDS session is not to be considered a database instance.\n",
     "\n",
-    "That means that all projections will go from AuraDB to GDS session, not from a local database.\n",
-    "Similarly, writing back will follow the same path back to AuraDB, and not to a local database.\n",
+    "That means that all projections will go from AuraDB to GDS session, not from a co-located database.\n",
+    "Similarly, writing back will follow the same path back to AuraDB, and not to a co-located database.\n",
     "\n",
     "## Implementation limitation\n",
     "\n",
     "As mentioned in the parenthesis above, we do make use of existing AuraDS infrastructure to host the GDS sessions.\n",
-    "Due to that fact, there actually is a local database, but we try to not expose its Bolt URI, in an attempt to prohibit users adding data to that database. "
+    "Due to that fact, there actually is a co-located database, but we try to not expose its Bolt URI, in an attempt to prohibit users adding data to that database. "
    ]
   },
   {
@@ -241,25 +254,34 @@
   },
   {
    "cell_type": "markdown",
+   "metadata": {},
    "source": [
     "#### Creating a new session\n",
     "\n",
     "A user can create a new session by calling `sessions.create_gds`.\n",
     "A session is identified by a name and needs a password to be set. The password is necessary to reconnect to an existing session.\n",
-    "Additionally an instance size can be provided. Possible values are  `8GB`, `16GB`, `24GB` (`32GB`, `48GB`, `64GB`, `96GB` are not available in the testing environment)\n",
+    "Additionally an instance size can be provided. Possible values are  `8GB`, `16GB`, `24GB` (`32GB`, `48GB`, `64GB`, `96GB` are not available in the testing environment).\n",
     "\n",
-    "Creating a new session takes a few minutes to complete. We know that this is not ideal and the problem is even exaggerated in the development environment because we do not keep that many cloud instances running in order to safe on cost.  "
-   ],
-   "metadata": {
-    "collapsed": false
-   }
+    "Creating a new session takes a few minutes to complete. We know that this is not ideal and the problem is even exaggerated in the development environment because we do not keep that many cloud VMs running in order to keep costs low.\n",
+    "\n",
+    "💵💵💵💵💵💵\n",
+    "\n",
+    "💰💰💰💰💰💰\n",
+    "\n",
+    "💸💸💸💸💸💸\n",
+    "\n",
+    "NOTE: the creation of a session marks the start of billable activity.\n",
+    "Sessions are machines that run in the cloud, and they cost money.\n",
+    "This cost will accumulate for the lifetime of the session, which needs to be manually deleted."
+   ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
+    "# let's create a GDS session!\n",
     "gds = sessions.create_gds(\"pagerank-compute\", \"my-password\", \"8GB\")"
    ]
   },
@@ -268,7 +290,7 @@
    "metadata": {},
    "source": [
     "Alternatively it is possible to reconnect to an existing session.\n",
-    "This is especiially handy if the session ran a long computation and the client is disconnected."
+    "This is especially handy if the session ran a long computation and the client is disconnected."
    ]
   },
   {
@@ -289,19 +311,22 @@
     "In order to project graphs from an AuraDB instance into the GDS session we created a new projection method: `gds.graph.project.remoteDb`\n",
     "The projection works similar to Cypher projections V2 and is implemented as an Cypher Aggregation function.\n",
     "The Cypher query containing the projection function is executed on the AuraDB instance and the data it produces is transferred to the \n",
-    "GDS session instance via Arrow. \n",
+    "GDS session instance via an Arrow connection. \n",
     "\n",
     "There are two key differences between the remote projection and Cypher projections V2:\n",
     "\n",
     "1. In AuraDB, the aggregating function does not take a graph name as a parameter.\n",
-    "2. The aggregation function should only be called through the python client `gds.graph.project.remoteDb`\n",
+    "2. The aggregation function should only be called through the GDS Python Client endpoint `gds.graph.project.remoteDb`\n",
     "\n",
     "### Limitations\n",
     "\n",
     "The aggregation function is currently limited to projecting homogeneous graph schemas. \n",
     "That means that all nodes/relationships will have the same property keys regardless of their labels or type. \n",
     "The caller of the aggregation function must ensure to supply all possible properties for each node or relationship. Null values are not supported.\n",
-    "\n"
+    "\n",
+    "The example data in this notebook contains only `User` nodes with `age` properties.\n",
+    "If there are also `Product` nodes with `cost` properties then we would need to add placeholder `cost` and `age` properties on the `User` and `Product` nodes, respectively.\n",
+    "This is a limitation we will attempt to address.\n"
    ]
   },
   {
@@ -367,27 +392,53 @@
   },
   {
    "cell_type": "markdown",
+   "metadata": {},
    "source": [
     "# Writing back to AuraDB\n",
     "\n",
-    "The sessions in-memory graph was projected from data in AuraDB.\n",
-    "Write back operations should thus also persist the data back to AuraDB.\n",
+    "The session's in-memory graph was projected from data in AuraDB.\n",
+    "Write back operations will thus persist the data back to the same AuraDB.\n",
     "\n",
-    "When calling any write operations the python client will automatically use the new remote write back functionality so that no API changes are necessary."
-   ],
-   "metadata": {
-    "collapsed": false
-   }
+    "When calling any write operations the python client will automatically use the new remote write back functionality so that no API changes are necessary.\n",
+    "\n",
+    "The AuraDB coordinates are not stored in the GDS session, but in the client.\n",
+    "Thus, it is important to set up the AuraSessions object with the DB credentials that identify the correct database from which the projection came."
+   ]
   },
   {
    "cell_type": "code",
    "execution_count": null,
    "metadata": {},
    "outputs": [],
    "source": [
+    "# if this fails once with some error like \"unable to retrieve routing table\"\n",
+    "# then run it again. this is a transient error with a stale server cache.\n",
     "gds.graph.nodeProperties.write(G, \"pagerank\")"
    ]
   },
+  {
+   "cell_type": "markdown",
+   "metadata": {},
+   "source": [
+    "Of course, we can just use `.write` modes as well:"
+   ]
+  },
+  {
+   "cell_type": "code",
+   "execution_count": null,
+   "metadata": {},
+   "outputs": [],
+   "source": [
+    "gds.fastRP.write(\n",
+    "    G,\n",
+    "    writeProperty=\"fastRP\",\n",
+    "    embeddingDimension=64,\n",
+    "    featureProperties=[\"pagerank\"],\n",
+    "    propertyRatio=0.2,\n",
+    "    nodeSelfInfluence=0.2,\n",
+    ")"
+   ]
+  },
   {
    "cell_type": "markdown",
    "metadata": {},
@@ -402,7 +453,14 @@
    "metadata": {},
    "outputs": [],
    "source": [
-    "gds.run_cypher(\"MATCH (u:User) RETURN u.pagerank\")"
+    "gds.run_cypher(\n",
+    "    \"\"\"\n",
+    "    MATCH (u:User) \n",
+    "    RETURN u.id, u.age, u.fastRP, u.pagerank AS rank \n",
+    "     ORDER BY rank DESC\n",
+    "     LIMIT 5\n",
+    "    \"\"\"\n",
+    ")"
    ]
   },
   {
@@ -412,10 +470,16 @@
     "# Closing the session\n",
     "\n",
     "Generally we intend for the sessions to only live for the time it takes to run a single workload.\n",
-    "If the same workload needs to be re-run, for example to wor with updated data, a new session would be created.\n",
+    "If the same workload needs to be re-run, for example to work with updated data, a new session would be created.\n",
+    "\n",
+    "💵💵💵💵💵💵\n",
+    "\n",
+    "💰💰💰💰💰💰\n",
+    "\n",
+    "💸💸💸💸💸💸\n",
     "\n",
-    "The `session.delete_gds` operations will close the session and release all resources associated with it.\n",
-    "It is important to note, that until this command was called the customer will be charged for the AuraDS instance that is used to host the session."
+    "The `session.delete_gds` operation will delete the session and release all resources associated with it.\n",
+    "It is important to note, that until this command was called the customer will be charged for the costs associated with hosting the session instance."
    ]
   },
   {
@@ -424,6 +488,8 @@
    "metadata": {},
    "outputs": [],
    "source": [
+    "# this will return True if it did delete something\n",
+    "# it will return False otherwise, but it will not normally fail\n",
     "sessions.delete_gds(\"pagerank-compute\")"
    ]
   }

requirements/dev/dev.txt

@@ -5,11 +5,8 @@ isort == 5.12.0
 mypy == 1.5.1
 nbconvert == 7.6.0
 pandas-stubs == 2.0.3.230814
-pytest == 7.4.0
 pytest-annotate == 1.0.5
 tox == 4.11.3
 types-setuptools == 68.1.0.1
 sphinx == 7.2.6
-requests_mock == 1.11.0
-pytest_mock == 3.12.0
 types-requests