similarity notebook to compare link prediction algos (#3868)

New notebook to compare link prediction 
Dependent on dining_prefs being added to datasets api in PR #3866

Authors:
  - Don Acosta (https://github.com/acostadon)

Approvers:
  - Brad Rees (https://github.com/BradReesWork)

URL: #3868

notebooks/algorithms/link_prediction/similarity_combined.ipynb

@@ -0,0 +1,217 @@
+{
+ "cells": [
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "# Similarity Compared\n",
+ "----\n",
+ "\n",
+ "In this notebook, we will compute vertex similarity scores using the various cuGraph algorithms. We will then compare the similarities scores in tables.\n",
+ "\n",
+ "| Author Credit | Date | Update | cuGraph Version | Test Hardware |\n",
+ "| --------------|------------|------------------|-----------------|-----------------------|\n",
+ "| Don Acosta | 09/25/2023 | created | 23.10 nightly | AMPERE A6000 CUDA 11.7|\n",
+ "\n",
+ "\n",
+ "**Note: On large graphs these algorithms can take prohibitive time or memory. The notebook will show how to run on defined pairs instead.**\n",
+ "\n",
+ "The Similarity algorithms in cuGraph use different methods to compare pairs of vertices. All of them use the intersection of the set of adjacent nodes for the set overlap. However each of the three algorithms differ on the denominator to determine the similarity coefficients. All three are normalized between zero and one. where zero is no overlap at all and one means identical adjacencies.\n",
+ "\n",
+ "__Jaccard Similarity__<br>\n",
+ "The [Jaccard similarity](https://en.wikipedia.org/wiki/Jaccard_index) measure was developed by botonist, Paul Jaccard who used the measure to compare plant species. His work popularized the measure's use in in other fields as well.\n",
+ "\n",
+ "It can be expressed as:<br>\n",
+ "$\\text{Jaccard similarity} = \\frac{|A \\cap B|}{|A \\cup B|}$\n",
+ "\n",
+ "__Overlap Similarity__<br>\n",
+ "The [Overlap Similarity](https://en.wikipedia.org/wiki/Overlap_coefficient) is also known as the Szymkiewicz–Simpson coefficient. It is often used to compare binary and categorical data in the fields of Genome analysis, recommender systems and anomaly detection. It differs from the Jaccard measure above in that it uses the size of the smaller of the two set sizes as the denominator.\n",
+ "\n",
+ "It can be expressed as\n",
+ "\n",
+ "$oc(A,B)=\\frac{|A|\\cap|B|}{min(|A|,|B|)}$\n",
+ "\n",
+ "__Sørensen-Dice Coefficient__<br>\n",
+ "The [Sørensen coefficient](https://en.wikipedia.org/wiki/S%C3%B8rensen%E2%80%93Dice_coefficient#) is known as the Sørensen-Dice coefficient. It was independently developed for use by botonists Lee Raymond Dice and Thorvald Sørensen. Although originating in the field of Botony, the coefficient is now used in computer vision, Natural Language Processing(NLP) and Data Mining among other fields.\n",
+ "It differs from Jaccard and Overlap in that the calculation doubles the intersection size and divides it by the sum of the two set sizes.\n",
+ "\n",
+ "It can be expressed as\n",
+ "\n",
+ "Sørensen coefficient = $\\left(2 * |A \\cap B| \\right) \\over \\left(|A| + |B| \\right)$\n",
+ "\n"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "---\n",
+ "# Now for the code !"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Load the required dependencies."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "import cugraph\n",
+ "from cugraph.datasets import dining_prefs\n",
+ "# only needed to display results in a table \n",
+ "from IPython.display import display_html "
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Function that calls all the cuGraph similarity/link prediction algorithms "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "def compute_similarity(G,pairs=None):\n",
+ " _jdf = cugraph.jaccard(G,pairs)\n",
+ " _jdf2 = _jdf[ (_jdf['first'] != _jdf['second'] ) ]\n",
+ " _odf = cugraph.overlap(G,pairs)\n",
+ " _odf2 = _odf[ (_odf['first'] != _odf['second'] ) ]\n",
+ " _sdf = cugraph.sorensen_coefficient(G,pairs)\n",
+ " _sdf2 = _sdf[ (_sdf['first'] != _sdf['second'] ) ]\n",
+ " return _jdf2, _odf2, _sdf2"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Function to put all the results in a convenient table"
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# Print function\n",
+ "def print_similarity(jdf,odf,sdf,num_records=5):\n",
+ "\n",
+ " js_top = jdf.sort_values(by='jaccard_coeff', ascending=False).head(num_records).to_pandas()\n",
+ " os_top = odf.sort_values(by='overlap_coeff', ascending=False).head(num_records).to_pandas()\n",
+ " ss_top = sdf.sort_values(by='sorensen_coeff', ascending=False).head(num_records).to_pandas()\n",
+ " \n",
+ " df1_styler = js_top.style.set_table_attributes(\"style='display:inline'\").set_caption('Jaccard').hide(axis='index')\n",
+ " df2_styler = os_top.style.set_table_attributes(\"style='display:inline'\").set_caption('Overlap').hide(axis='index')\n",
+ " df3_styler = ss_top.style.set_table_attributes(\"style='display:inline'\").set_caption('Sørensen').hide(axis='index')\n",
+ "\n",
+ " display_html(df1_styler._repr_html_()+df2_styler._repr_html_()+df3_styler._repr_html_(), raw=True)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Create the graph from the Dining preferences data set."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "G = dining_prefs.get_graph(download=True)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Run the three similarity Algorithms and print out the five links with the highest scores."
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "jdf, odf, sdf = compute_similarity(G)\n",
+ "print_similarity(jdf,odf,sdf)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "Now find the the complete set of two-hop neigbors and compare them instead of just using the existing one-hop edges. In a larger graph, this will run considerably faster since the default "
+ ]
+ },
+ {
+ "cell_type": "code",
+ "execution_count": null,
+ "metadata": {},
+ "outputs": [],
+ "source": [
+ "# this cugraph algorithm pulls a set containing every pair of vertices\n",
+ "# that are within 2-hops of each other\n",
+ "two_hops_pairs = G.get_two_hop_neighbors()\n",
+ "\n",
+ "jdf_hops, odf_hops, sdf_hops = compute_similarity(G,pairs=two_hops_pairs)\n",
+ "print_similarity(jdf_hops,odf_hops,sdf_hops)"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "### It's that easy with cuGraph"
+ ]
+ },
+ {
+ "cell_type": "markdown",
+ "metadata": {},
+ "source": [
+ "----\n",
+ "Copyright (c) 2023, NVIDIA CORPORATION.\n",
+ "\n",
+ "Licensed under the Apache License, Version 2.0 (the \"License\"); you may not use this file except in compliance with the License. You may obtain a copy of the License at http://www.apache.org/licenses/LICENSE-2.0\n",
+ "\n",
+ "Unless required by applicable law or agreed to in writing, software distributed under the License is distributed on an \"AS IS\" BASIS, WITHOUT WARRANTIES OR CONDITIONS OF ANY KIND, either express or implied. See the License for the specific language governing permissions and limitations under the License."
+ ]
+ }
+ ],
+ "metadata": {
+ "kernelspec": {
+ "display_name": "cugraph_0802",
+ "language": "python",
+ "name": "python3"
+ },
+ "language_info": {
+ "codemirror_mode": {
+ "name": "ipython",
+ "version": 3
+ },
+ "file_extension": ".py",
+ "mimetype": "text/x-python",
+ "name": "python",
+ "nbconvert_exporter": "python",
+ "pygments_lexer": "ipython3",
+ "version": "3.10.12"
+ },
+ "orig_nbformat": 4
+ },
+ "nbformat": 4,
+ "nbformat_minor": 2
+}