tutorial_hinsage_ans.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# HinSAGE Tutorial"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Movie recommendation as edge regression with [HinSAGE](https://stellargraph.readthedocs.io/en/stable/api.html#module-stellargraph.layer.hinsage).\n",
    "\n",
    "We first build an (undirected) bipartite graph with users/movies as (heterogeneous) nodes, and the ratings of (user, movie) pairs as edge weights, then learn the node embeddings, and recommend movies for users by regressing the weight of a given edge, i.e., a (user, movie) pair."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline\n",
    "\n",
    "import os\n",
    "import json\n",
    "import random\n",
    "import numpy as np\n",
    "import pandas as pd\n",
    "import multiprocessing\n",
    "import matplotlib.pyplot as plt"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Requirement already up-to-date: gast==0.2.2 in /home/dawei/apps/miniconda3.7/lib/python3.7/site-packages (0.2.2)\r\n"
     ]
    }
   ],
   "source": [
    "# discard warnings\n",
    "import warnings\n",
    "warnings.filterwarnings('ignore', category=FutureWarning)\n",
    "\n",
    "from tensorflow.python.util import deprecation\n",
    "deprecation._PRINT_DEPRECATION_WARNINGS = False\n",
    "\n",
    "!pip install -U 'gast==0.2.2'"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "from sklearn import preprocessing, feature_extraction\n",
    "from sklearn.model_selection import train_test_split\n",
    "from sklearn.metrics import mean_squared_error\n",
    "\n",
    "import tensorflow as tf\n",
    "import tensorflow.keras.backend as K\n",
    "from tensorflow import keras\n",
    "\n",
    "import stellargraph as sg\n",
    "from stellargraph.mapper import HinSAGELinkGenerator\n",
    "from stellargraph.layer import HinSAGE, link_regression\n",
    "\n",
    "from utils import ingest_features, ingest_graph, add_features_to_nodes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [],
   "source": [
    "# set random seed\n",
    "SEED = 101\n",
    "random.seed(SEED)\n",
    "np.random.seed(SEED)\n",
    "tf.set_random_seed(SEED)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Load the MovieLens 100k Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load the [movielens 100k dataset](http://files.grouplens.org/datasets/movielens/ml-100k.zip) and build a bipartite graph with users/movies as nodes, and the ratings of (user, movie) pairs as edge weights. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [],
   "source": [
    "data_dir = os.path.join('data', 'ml-100k')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Graph statistics: 943 users, 1682 movies, 100000 ratings\n"
     ]
    }
   ],
   "source": [
    "config = json.load(open('ml-100k-config.json', 'r'))\n",
    "Gnx, id_map, inv_id_map = ingest_graph(data_dir, config)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load user features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "user_features = ingest_features(data_dir, config, node_type='users')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Preprocess user features: normalising user `age`, and performing one-hot encoding for `gender` and `job`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "feature_names = ['age', 'gender', 'job']\n",
    "feature_encoding = feature_extraction.DictVectorizer(sparse=False, dtype=np.float)\n",
    "user_features_transformed = feature_encoding.fit_transform(user_features[feature_names].to_dict('records'))\n",
    "user_features_transformed[:, 0] = preprocessing.scale(user_features_transformed[:, 0])  # rescale ages\n",
    "user_features = pd.DataFrame(user_features_transformed, index=user_features.index, dtype=np.float)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Load movie features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [],
   "source": [
    "movie_features = ingest_features(data_dir, config, node_type='movies')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Add user and movie features to the graph"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [],
   "source": [
    "Gnx = add_features_to_nodes(Gnx, inv_id_map, user_features, movie_features)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Split edges into train and test sets"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [],
   "source": [
    "edges_train, edges_test = train_test_split(list(Gnx.edges(data=True)), train_size=0.7, test_size=0.3)\n",
    "\n",
    "edgelist_train = [(e[0],e[1]) for e in edges_train]\n",
    "edgelist_test = [(e[0],e[1]) for e in edges_test]\n",
    "\n",
    "labels_train = [e[2]['score'] for e in edges_train]\n",
    "labels_test = [e[2]['score'] for e in edges_test]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Learn node embedding using supervised HinSAGE"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create an undirected stellargraph model with node features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "G = sg.StellarGraph(Gnx, node_features='feature')\n",
    "batch_size = 32\n",
    "num_samples = [10, 5]  # sizes of the 1- and 2-hop neighbour samples"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create the generators to feed data from the graph to the Keras model."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [],
   "source": [
    "link_gen = HinSAGELinkGenerator(G, batch_size, num_samples)\n",
    "train_gen = link_gen.flow(edgelist_train, labels_train, shuffle=True)\n",
    "test_gen = link_gen.flow(edgelist_test, labels_test, shuffle=False)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 1\n",
    "\n",
    "Create a [HinSAGE](https://stellargraph.readthedocs.io/en/stable/api.html#stellargraph.layer.hinsage.HinSAGE) model with:\n",
    "- 2 hidden layers\n",
    "- the size of both layers is 32\n",
    "- the link generator created above\n",
    "- no dropout"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [],
   "source": [
    "# hinsage = # YOUR_CODE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [],
   "source": [
    "hinsage = HinSAGE(layer_sizes=[32, 32], \n",
    "                  generator=train_gen, \n",
    "                  bias=True, \n",
    "                  dropout=0.0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {
    "scrolled": true
   },
   "outputs": [],
   "source": [
    "x_inp, x_out = hinsage.build()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "### Exercise 2\n",
    "\n",
    "Create a regression layer using [link_regression](https://stellargraph.readthedocs.io/en/stable/api.html#stellargraph.layer.link_inference.link_regression) which uses the concatenated embeddings of nodes (i.e., user/movie) as the edge embedding."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 17,
   "metadata": {},
   "outputs": [],
   "source": [
    "# pred_layer = # YOUR_CODE"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 18,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "link_regression: using 'concat' method to combine node embeddings into edge embeddings\n"
     ]
    }
   ],
   "source": [
    "pred_layer = link_regression(edge_embedding_method='concat')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 19,
   "metadata": {},
   "outputs": [],
   "source": [
    "prediction = pred_layer(x_out)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Create a Keras model that combines node embedding learning and link regression layers, then train it."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 20,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Epoch 1/3\n",
      "2188/2188 [==============================] - 35s 16ms/step - loss: 1.3699 - root_mean_squared_error: 1.1357\n",
      "Epoch 2/3\n",
      "2188/2188 [==============================] - 33s 15ms/step - loss: 1.1298 - root_mean_squared_error: 1.0556\n",
      "Epoch 3/3\n",
      "2188/2188 [==============================] - 33s 15ms/step - loss: 1.1148 - root_mean_squared_error: 1.0484\n"
     ]
    }
   ],
   "source": [
    "model = keras.Model(inputs=x_inp, outputs=prediction)\n",
    "\n",
    "def root_mean_squared_error(y_true, y_pred): \n",
    "    return K.sqrt(K.mean((y_true - y_pred) ** 2))\n",
    "\n",
    "model.compile(\n",
    "    optimizer=keras.optimizers.Adam(lr=1e-3),\n",
    "    loss=keras.losses.mean_squared_error,\n",
    "    metrics=[root_mean_squared_error],\n",
    ")\n",
    "\n",
    "_ = model.fit_generator(\n",
    "          train_gen,\n",
    "          epochs=3,\n",
    "          verbose=1,\n",
    "          shuffle=True,\n",
    "          workers=multiprocessing.cpu_count()//2)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Make recommendations"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Baseline I**\n",
    "\n",
    "Predict the rating of a movie using the average of observed ratings of that movie"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 21,
   "metadata": {},
   "outputs": [],
   "source": [
    "ratings_dict = dict()\n",
    "ratings_total = 0\n",
    "\n",
    "for e in edges_train:\n",
    "    mid = e[2]['mId']\n",
    "    score = e[2]['score']\n",
    "    ratings_total += score\n",
    "    try:\n",
    "        ratings_dict[mid].append(score)\n",
    "    except KeyError:\n",
    "        ratings_dict[mid] = [score]\n",
    "        \n",
    "rating_mean = ratings_total / len(edges_train)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "y_pred_baseline = []\n",
    "\n",
    "for e in edges_test:\n",
    "    mid = e[2]['mId']\n",
    "    if mid in ratings_dict:\n",
    "        pred = np.mean(ratings_dict[mid]) \n",
    "    else:\n",
    "        pred = rating_mean\n",
    "    y_pred_baseline.append(pred)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 23,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "938/938 [==============================] - 11s 12ms/step\n"
     ]
    }
   ],
   "source": [
    "y_true = labels_test\n",
    "y_pred = model.predict_generator(test_gen, verbose=1)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 24,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "RMSE of Baseline: 1.026\n",
      "RMSE of  HinSAGE: 1.056\n"
     ]
    }
   ],
   "source": [
    "print('RMSE of Baseline: %.3f' % np.sqrt(mean_squared_error(y_true, y_pred_baseline)))\n",
    "print('RMSE of  HinSAGE: %.3f' % np.sqrt(mean_squared_error(y_true, y_pred)))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}