examples/node_classification/injection_attack.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "82e9886a-1de2-422c-8fba-9302f09c91a7",
   "metadata": {},
   "source": [
    "# 1. Example of graph injection attack using GRB"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7837654b-f823-45e1-b80a-c1431f278ee4",
   "metadata": {},
   "source": [
    "GRB provides a unified evaluation scenario for fair comparisons between attacks and defenses. The scenario is **Black-box, Evasion, Inductive, Injection**. Take the case of a citation-graph classification system for example. The platform collects labeled data from previous papers and trains a GNN model. When a batch of new papers are submitted, it updates the graph and uses the trained model to predict labels for them.\n",
    "\n",
    "* **Black-box**: Both the attacker and the defender have no knowledge about the applied methods each other uses.\n",
    "* **Evasion**: GNNs are already trained in trusted data (e.g. authenticated users), which are untouched by the attackers but might have natural noises. Thus, attacks will only happen during the inference phase.\n",
    "* **Inductive**: GNNs are used to classify unseen data (e.g. new users), i.e. validation or test data are unseen during training, which requires GNNs to generalize to out of distribution data.\n",
    "* **Injection**: The attackers can only inject new nodes but not modify the target nodes directly. Since it is usually hard to hack into users' accounts and modify their profiles. However, it is easier to create fake accounts and connect them to existing users."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "id": "897373a0-691e-4c91-9eb3-1ff59798428d",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-24T16:50:58.218011Z",
     "iopub.status.busy": "2021-08-24T16:50:58.217712Z",
     "iopub.status.idle": "2021-08-24T16:50:59.290448Z",
     "shell.execute_reply": "2021-08-24T16:50:59.289898Z",
     "shell.execute_reply.started": "2021-08-24T16:50:58.217935Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "import os\n",
    "import torch\n",
    "import grb.utils as utils"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "36544446-57d0-400d-b948-39acef59dd9b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-11T11:59:56.178453Z",
     "iopub.status.busy": "2021-08-11T11:59:56.178207Z",
     "iopub.status.idle": "2021-08-11T11:59:56.181058Z",
     "shell.execute_reply": "2021-08-11T11:59:56.180559Z",
     "shell.execute_reply.started": "2021-08-11T11:59:56.178427Z"
    },
    "tags": []
   },
   "source": [
    "## 1.1. Load Dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "ac3e7f1d-7213-4f33-91fc-3a1b514d18e5",
   "metadata": {},
   "source": [
    "GRB datasets are named by the prefix *grb-*. There are four *mode* ('easy', 'medium', 'hard', 'full') for test set, representing different average degrees of test nodes, thus different difficulty for attacking them. The node features are processed by *arctan* normalization (first standardization then arctan function), which makes node features fall in the same scale."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "id": "143458da-bb41-485e-a11c-790cd920bafe",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-24T16:51:00.712670Z",
     "iopub.status.busy": "2021-08-24T16:51:00.712437Z",
     "iopub.status.idle": "2021-08-24T16:51:00.805959Z",
     "shell.execute_reply": "2021-08-24T16:51:00.805500Z",
     "shell.execute_reply.started": "2021-08-24T16:51:00.712646Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Dataset 'grb-cora' loaded.\n",
      "    Number of nodes: 2680\n",
      "    Number of edges: 5148\n",
      "    Number of features: 302\n",
      "    Number of classes: 7\n",
      "    Number of train samples: 1608\n",
      "    Number of val samples: 268\n",
      "    Number of test samples: 804\n",
      "    Dataset mode: full\n",
      "    Feature range: [-0.9406, 0.9430]\n"
     ]
    }
   ],
   "source": [
    "from grb.dataset import Dataset\n",
    "\n",
    "dataset_name = 'grb-cora'\n",
    "dataset = Dataset(name=dataset_name, \n",
    "                  data_dir=\"../../data/\",\n",
    "                  mode='full',\n",
    "                  feat_norm='arctan')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "id": "efa6fb77-4cdb-4cfc-b0d6-90d94948a286",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-21T11:57:49.015592Z",
     "iopub.status.busy": "2021-08-21T11:57:49.015408Z",
     "iopub.status.idle": "2021-08-21T11:57:49.018560Z",
     "shell.execute_reply": "2021-08-21T11:57:49.018124Z",
     "shell.execute_reply.started": "2021-08-21T11:57:49.015568Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "adj = dataset.adj\n",
    "features = dataset.features\n",
    "labels = dataset.labels\n",
    "num_features = dataset.num_features\n",
    "num_classes = dataset.num_classes\n",
    "test_mask = dataset.test_mask"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "b8be88ab-d574-4f6b-9344-9bb6298cc0d0",
   "metadata": {},
   "source": [
    "## 1.2. Graph Injection Attack"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f339cd2-c0ea-489a-a192-e1ef8527da5a",
   "metadata": {},
   "source": [
    "For graph injection attack under the black box setting, we need to first train a surrogate model, then transfer the generated attack nodes to a target model. Note that the attacker doesn't have any information of the target model, neither the model architecture nor the parameters. Here is an example of training GCN as the surrogate model, and transfer to other models. "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "90ae18fa-e7a0-436f-9a9c-770fe7f3ef30",
   "metadata": {
    "tags": []
   },
   "source": [
    "### 1.2.1 Train surrogate model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "id": "ae567711-1820-41ce-b0ad-e267d0d14053",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-21T11:57:50.216140Z",
     "iopub.status.busy": "2021-08-21T11:57:50.215892Z",
     "iopub.status.idle": "2021-08-21T11:57:50.224565Z",
     "shell.execute_reply": "2021-08-21T11:57:50.224055Z",
     "shell.execute_reply.started": "2021-08-21T11:57:50.216114Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "GCN(\n",
      "  (layers): ModuleList(\n",
      "    (0): GCNConv(\n",
      "      (linear): Linear(in_features=302, out_features=64, bias=True)\n",
      "      (dropout): Dropout(p=0.5, inplace=False)\n",
      "    )\n",
      "    (1): GCNConv(\n",
      "      (linear): Linear(in_features=64, out_features=7, bias=True)\n",
      "    )\n",
      "  )\n",
      ")\n"
     ]
    }
   ],
   "source": [
    "from grb.model.torch import GCN\n",
    "from grb.utils.normalize import GCNAdjNorm\n",
    "\n",
    "model_name = \"gcn\"\n",
    "model_sur = GCN(in_features=dataset.num_features,\n",
    "                out_features=dataset.num_classes,\n",
    "                hidden_features=64, \n",
    "                n_layers=2,\n",
    "                adj_norm_func=GCNAdjNorm,\n",
    "                layer_norm=False,\n",
    "                residual=False,\n",
    "                dropout=0.5)\n",
    "print(model_sur)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "id": "633f82da-0f64-46d4-b119-225eda2ef10c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-21T11:57:50.795857Z",
     "iopub.status.busy": "2021-08-21T11:57:50.795675Z",
     "iopub.status.idle": "2021-08-21T11:57:50.798556Z",
     "shell.execute_reply": "2021-08-21T11:57:50.798094Z",
     "shell.execute_reply.started": "2021-08-21T11:57:50.795839Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "save_dir = \"./saved_models/{}/{}\".format(dataset_name, model_name)\n",
    "save_name = \"model_sur.pt\"\n",
    "device = \"cuda:0\"\n",
    "feat_norm = None\n",
    "train_mode = \"inductive\"  # \"transductive\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "id": "a2ee402e-003b-4101-9521-b666d8de1137",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-21T11:57:50.995447Z",
     "iopub.status.busy": "2021-08-21T11:57:50.995336Z",
     "iopub.status.idle": "2021-08-21T11:57:52.643483Z",
     "shell.execute_reply": "2021-08-21T11:57:52.642870Z",
     "shell.execute_reply.started": "2021-08-21T11:57:50.995433Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from grb.trainer.trainer import Trainer\n",
    "\n",
    "trainer = Trainer(dataset=dataset, \n",
    "                  optimizer=torch.optim.Adam(model_sur.parameters(), lr=0.01),\n",
    "                  loss=torch.nn.functional.cross_entropy,\n",
    "                  lr_scheduler=False,\n",
    "                  early_stop=True,\n",
    "                  early_stop_patience=500,\n",
    "                  feat_norm=feat_norm,\n",
    "                  device=device)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "id": "392627fe-c902-4e74-9b81-896a5e6ad135",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-21T11:57:52.644380Z",
     "iopub.status.busy": "2021-08-21T11:57:52.644278Z",
     "iopub.status.idle": "2021-08-21T11:57:55.332239Z",
     "shell.execute_reply": "2021-08-21T11:57:55.331692Z",
     "shell.execute_reply.started": "2021-08-21T11:57:52.644365Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "60ac8ceee3344a909de1163939b8c1f7",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/2000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Training early stopped. Best validation score: 0.8396\n",
      "Training runtime: 2.6830.\n"
     ]
    }
   ],
   "source": [
    "trainer.train(model=model_sur, \n",
    "              n_epoch=2000,\n",
    "              eval_every=1,\n",
    "              save_after=0,\n",
    "              save_dir=save_dir,\n",
    "              save_name=save_name,\n",
    "              train_mode=train_mode,\n",
    "              verbose=False)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "id": "c82c515e-57ad-4c8e-85c3-aedbe45182ed",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-21T11:57:55.333159Z",
     "iopub.status.busy": "2021-08-21T11:57:55.333066Z",
     "iopub.status.idle": "2021-08-21T11:57:55.340835Z",
     "shell.execute_reply": "2021-08-21T11:57:55.340287Z",
     "shell.execute_reply.started": "2021-08-21T11:57:55.333147Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test score of surrogate model: 0.8296\n"
     ]
    }
   ],
   "source": [
    "# by trainer\n",
    "test_score = trainer.evaluate(model_sur, dataset.test_mask)\n",
    "print(\"Test score of surrogate model: {:.4f}\".format(test_score))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "fa5f63f0-6671-4852-966e-6a2263e10d1b",
   "metadata": {},
   "source": [
    "### 1.2.2. Injection Attack"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "678ef8d4-7ef2-4d8d-8d7e-3d9bb2603fcc",
   "metadata": {},
   "source": [
    "**Rules and constraints for attackers**: they have knowledge about the entire graph (including all nodes, edges and labels, excluding labels of the test nodes to attack), but do NOT have knowledge about the target model or the defense mechanism; they are allowed to inject a limited number of new nodes with limited edges, but are NOT allowed to modify the original graph; they are allowed to generate features of injected nodes as long as they remain unnoticeable by defenders (e.g. nodes with features that exceed the range can be easily detected); they are allowed to get the classification results from the target model through limited number of queries."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6f0b4899-9c28-41d2-8feb-a6336493da43",
   "metadata": {},
   "source": [
    "#### FGSM (Fast Gradient Sign Method)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "7280c06b-09c7-4dcc-a794-5a9c16b3e92a",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-21T11:45:08.815399Z",
     "iopub.status.busy": "2021-08-21T11:45:08.815143Z",
     "iopub.status.idle": "2021-08-21T11:45:08.828526Z",
     "shell.execute_reply": "2021-08-21T11:45:08.828072Z",
     "shell.execute_reply.started": "2021-08-21T11:45:08.815374Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from grb.attack.injection import FGSM\n",
    "\n",
    "attack = FGSM(epsilon=0.01,\n",
    "              n_epoch=1000,\n",
    "              n_inject_max=100,\n",
    "              n_edge_max=200,\n",
    "              feat_lim_min=-1,\n",
    "              feat_lim_max=1,\n",
    "              device=device)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "378db821-f808-4de6-88b9-f48d3a8e7701",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-21T11:24:30.797750Z",
     "iopub.status.busy": "2021-08-21T11:24:30.797505Z",
     "iopub.status.idle": "2021-08-21T11:24:30.800750Z",
     "shell.execute_reply": "2021-08-21T11:24:30.800176Z",
     "shell.execute_reply.started": "2021-08-21T11:24:30.797726Z"
    }
   },
   "source": [
    "#### PGD (Projected Gradient Descent)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "b4df1217-7567-4f9e-bb10-7ae8ab137a5d",
   "metadata": {
    "ExecuteTime": {
     "end_time": "2021-04-01T13:04:28.728188Z",
     "start_time": "2021-04-01T13:04:28.725094Z"
    },
    "execution": {
     "iopub.execute_input": "2021-08-21T11:45:10.465113Z",
     "iopub.status.busy": "2021-08-21T11:45:10.464872Z",
     "iopub.status.idle": "2021-08-21T11:45:10.468750Z",
     "shell.execute_reply": "2021-08-21T11:45:10.468187Z",
     "shell.execute_reply.started": "2021-08-21T11:45:10.465090Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from grb.attack.injection import PGD\n",
    "\n",
    "attack = PGD(epsilon=0.01,\n",
    "             n_epoch=1000,\n",
    "             n_inject_max=100,\n",
    "             n_edge_max=200,\n",
    "             feat_lim_min=-1,\n",
    "             feat_lim_max=1,\n",
    "             device=device)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d384cba9-9a7d-483a-ad16-d480f0fe1d3b",
   "metadata": {},
   "source": [
    "#### RAND (Random)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "94f82fe2-7f74-4beb-9924-618e2f57328b",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-21T11:45:11.269854Z",
     "iopub.status.busy": "2021-08-21T11:45:11.269612Z",
     "iopub.status.idle": "2021-08-21T11:45:11.273337Z",
     "shell.execute_reply": "2021-08-21T11:45:11.272780Z",
     "shell.execute_reply.started": "2021-08-21T11:45:11.269831Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from grb.attack.injection import RAND\n",
    "\n",
    "attack = RAND(n_inject_max=100,\n",
    "              n_edge_max=200,\n",
    "              feat_lim_min=-1,\n",
    "              feat_lim_max=1,\n",
    "              device=device)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "570d52a5-8dac-4b99-8b02-7eaa711c0db5",
   "metadata": {},
   "source": [
    "#### SPEIT"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "0efb1867-b926-4f69-9188-41c050c942a1",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-21T11:45:12.255454Z",
     "iopub.status.busy": "2021-08-21T11:45:12.255207Z",
     "iopub.status.idle": "2021-08-21T11:45:12.259009Z",
     "shell.execute_reply": "2021-08-21T11:45:12.258508Z",
     "shell.execute_reply.started": "2021-08-21T11:45:12.255429Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from grb.attack.injection import SPEIT\n",
    "\n",
    "attack = SPEIT(lr=0.01,\n",
    "               n_epoch=1000,\n",
    "               n_inject_max=100,\n",
    "               n_edge_max=200,\n",
    "               feat_lim_min=-1,\n",
    "               feat_lim_max=1,\n",
    "               device=device)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "50e8fb41-c0b5-4f21-ad9a-ba8b426908b9",
   "metadata": {},
   "source": [
    "#### TDGIA (Topological Defective Graph Injection Attack)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "id": "5890b664-4370-4a61-9f65-b163999fe8e3",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-21T11:58:00.256516Z",
     "iopub.status.busy": "2021-08-21T11:58:00.256276Z",
     "iopub.status.idle": "2021-08-21T11:58:00.265549Z",
     "shell.execute_reply": "2021-08-21T11:58:00.265155Z",
     "shell.execute_reply.started": "2021-08-21T11:58:00.256491Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "from grb.attack.injection import TDGIA\n",
    "\n",
    "attack = TDGIA(lr=0.01,\n",
    "               n_epoch=1000,\n",
    "               n_inject_max=100,\n",
    "               n_edge_max=200,\n",
    "               feat_lim_min=-1,\n",
    "               feat_lim_max=1,\n",
    "               device=device)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "987373f6-ea23-4ed9-91e3-e1b9c6491fcb",
   "metadata": {},
   "source": [
    "### 1.2.3. Apply injection attack "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "id": "dd425914-768b-4eb3-87c0-9bfa79ddb148",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-21T11:58:01.136368Z",
     "iopub.status.busy": "2021-08-21T11:58:01.136122Z",
     "iopub.status.idle": "2021-08-21T11:58:19.800881Z",
     "shell.execute_reply": "2021-08-21T11:58:19.800509Z",
     "shell.execute_reply.started": "2021-08-21T11:58:01.136344Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Attacking: Sequential inject 20/100 nodes\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "aadd207b2d62470398a7484ee1d694ac",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Attacking: Sequential inject 40/100 nodes\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e18d730c14534eeb96c57e492a01bbb8",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Attacking: Sequential inject 60/100 nodes\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "f0b13bf5c29d4a3a9e7c37454a35284f",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Attacking: Sequential inject 80/100 nodes\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "e06a804b543147caab3e7f4298b2e240",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Attacking: Sequential inject 100/100 nodes\n"
     ]
    },
    {
     "data": {
      "application/vnd.jupyter.widget-view+json": {
       "model_id": "a4f43efc9aff411ba8f790758e82ac2d",
       "version_major": 2,
       "version_minor": 0
      },
      "text/plain": [
       "  0%|          | 0/1000 [00:00<?, ?it/s]"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Attack runtime: 18.6615.\n"
     ]
    }
   ],
   "source": [
    "adj_attack, features_attack = attack.attack(model=model_sur,\n",
    "                                            adj=adj,\n",
    "                                            features=features,\n",
    "                                            target_mask=test_mask,\n",
    "                                            adj_norm_func=model_sur.adj_norm_func)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "id": "bc530311-7bcf-4f1d-8728-0084157b2a52",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-21T11:58:20.689349Z",
     "iopub.status.busy": "2021-08-21T11:58:20.689105Z",
     "iopub.status.idle": "2021-08-21T11:58:20.701738Z",
     "shell.execute_reply": "2021-08-21T11:58:20.701297Z",
     "shell.execute_reply.started": "2021-08-21T11:58:20.689325Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test score after attack for surrogate model: 0.3296.\n"
     ]
    }
   ],
   "source": [
    "features_attacked = torch.cat([features.to(device), features_attack])\n",
    "test_score = utils.evaluate(model_sur, \n",
    "                            features=features_attacked,\n",
    "                            adj=adj_attack,\n",
    "                            labels=dataset.labels,\n",
    "                            adj_norm_func=model_sur.adj_norm_func,\n",
    "                            mask=dataset.test_mask,\n",
    "                            device=device)\n",
    "print(\"Test score after attack for surrogate model: {:.4f}.\".format(test_score))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "043ec115-3d21-4383-a32d-a28e94cbc991",
   "metadata": {
    "tags": []
   },
   "source": [
    "### 1.2.4. Transfer to target model"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "id": "da088acf-54d3-466d-ad37-9a9a8f6e29ae",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-21T11:58:22.259148Z",
     "iopub.status.busy": "2021-08-21T11:58:22.258897Z",
     "iopub.status.idle": "2021-08-21T11:58:22.262206Z",
     "shell.execute_reply": "2021-08-21T11:58:22.261770Z",
     "shell.execute_reply.started": "2021-08-21T11:58:22.259123Z"
    },
    "tags": []
   },
   "outputs": [],
   "source": [
    "model_name = \"gcn\"\n",
    "save_dir = \"./saved_models/{}/{}\".format(dataset_name, model_name)\n",
    "save_name = \"model.pt\"\n",
    "device = \"cuda:0\""
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "id": "b6248303-673f-43f3-9d73-aba127419785",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-21T11:58:22.791485Z",
     "iopub.status.busy": "2021-08-21T11:58:22.791274Z",
     "iopub.status.idle": "2021-08-21T11:58:22.799223Z",
     "shell.execute_reply": "2021-08-21T11:58:22.798824Z",
     "shell.execute_reply.started": "2021-08-21T11:58:22.791464Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "data": {
      "text/plain": [
       "GCN(\n",
       "  (layers): ModuleList(\n",
       "    (0): LayerNorm((302,), eps=1e-05, elementwise_affine=True)\n",
       "    (1): GCNConv(\n",
       "      (linear): Linear(in_features=302, out_features=64, bias=True)\n",
       "      (dropout): Dropout(p=0.5, inplace=False)\n",
       "    )\n",
       "    (2): LayerNorm((64,), eps=1e-05, elementwise_affine=True)\n",
       "    (3): GCNConv(\n",
       "      (linear): Linear(in_features=64, out_features=64, bias=True)\n",
       "      (dropout): Dropout(p=0.5, inplace=False)\n",
       "    )\n",
       "    (4): LayerNorm((64,), eps=1e-05, elementwise_affine=True)\n",
       "    (5): GCNConv(\n",
       "      (linear): Linear(in_features=64, out_features=7, bias=True)\n",
       "    )\n",
       "  )\n",
       ")"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "model = torch.load(os.path.join(save_dir, save_name))\n",
    "model = model.to(device)\n",
    "model.eval()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "id": "536c500e-afbd-49c5-b5f4-77644723ff2c",
   "metadata": {
    "execution": {
     "iopub.execute_input": "2021-08-21T11:58:25.496373Z",
     "iopub.status.busy": "2021-08-21T11:58:25.496134Z",
     "iopub.status.idle": "2021-08-21T11:58:25.509534Z",
     "shell.execute_reply": "2021-08-21T11:58:25.509196Z",
     "shell.execute_reply.started": "2021-08-21T11:58:25.496349Z"
    },
    "tags": []
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Test score after attack for target model: 0.5697.\n"
     ]
    }
   ],
   "source": [
    "test_score = utils.evaluate(model, \n",
    "                            features=features_attacked,\n",
    "                            adj=adj_attack,\n",
    "                            labels=dataset.labels,\n",
    "                            adj_norm_func=model.adj_norm_func,\n",
    "                            mask=dataset.test_mask,\n",
    "                            device=device)\n",
    "print(\"Test score after attack for target model: {:.4f}.\".format(test_score))"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.11"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}