Introduction_by_example.ipynb

{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Handling of Graphs"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "A graph is used to model pairwise relations (edges) between objects (nodes). A single graph in PyTorch Geometric is described by an instance of torch_geometric.data.Data, which holds the following attributes by default:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "그래프는 pairwise 관계(edges)를 사용한다. single graph는 '''torch_geometric.data.Data''' 에 설명되어있다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "- data.x\n",
    "    - Node feature matrix with shape [num_nodes, num_node_features]\n",
    "    - [노드 갯수, 특성의 차원]\n",
    "\n",
    "- data.edge_index\n",
    "    - Graph connectivity in COO format with shape [2, num_edges] and type torch.long\n",
    "    - 어떤 노드와 어떤 노드가 연결되어있는지 나타내는 특성인듯\n",
    "\n",
    "- data.edge_attr\n",
    "    - Edge feature matrix with shape [num_edges, num_edge_features]\n",
    "    - 엣지의 피쳐\n",
    "\n",
    "- data.y\n",
    "    - Target to train against (may have arbitrary shape), e.g., node-level targets of shape [num_nodes, *] or graph-level targets of shape [1, *]\n",
    "    - 학습시킬 것의 형태를 말하는 듯\n",
    "    \n",
    "- data.pos\n",
    "    - Node position matrix with shape [num_nodes, num_dimensions]\n",
    "    - 실제 위치를 나타내는듯?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "None of these attributes is required. In fact, the Data object is not even restricted to these attributes. We can, e.g., extend it by data.face to save the connectivity of triangles from a 3D mesh in a tensor with shape [3, num_faces] and type torch.long.\n",
    "\n",
    "이 특성들이 필요한 것은 아니다. 사실, Data 객체는 이 특성들에 제한되지않는다. 우리는 data.face를 통해서 [3, num_faces]의 크기를 가진 텐서의 3D mesh로부터 삼각형들의 연결성을 저장함으로써 그것을 확장시킬 수 있다(?)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We show a simple example of an unweighted and undirected graph with three nodes and four edges. Each node contains exactly one feature:\n",
    "\n",
    "- unweighted\n",
    "- undirected\n",
    "- one feature"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from torch_geometric.data import Data\n",
    "\n",
    "edge_index = torch.tensor([[0, 1, 1, 2],\n",
    "                           [1, 0, 2, 1]], dtype=torch.long)\n",
    "x = torch.tensor([[-1], [0], [1]], dtype=torch.float) #노드의 특성 여기선 1차원이니까 실수 하나\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![graph](img/graph.svg)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Data(edge_index=[2, 4], x=[3, 1])"
      ]
     },
     "execution_count": 2,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = Data(x=x, edge_index=edge_index)\n",
    "data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that '''edge_index''', i.e. the tensor defining the source and target nodes of all edges, is NOT a list of index tuples. If you want to write your indices this way, you should transpose and call '''contiguous''' on it before passing them to the data constructor:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "from torch_geometric.data import Data\n",
    "\n",
    "edge_index = torch.tensor([[0, 1],\n",
    "                           [1, 0],\n",
    "                           [1, 2],\n",
    "                           [2, 1]], dtype=torch.long)\n",
    "x = torch.tensor([[-1], [0], [1]], dtype=torch.float)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Data(edge_index=[2, 4], x=[3, 1])"
      ]
     },
     "execution_count": 4,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = Data(x=x, edge_index=edge_index.t().contiguous())\n",
    "data #위와 같은 방법인데, edge_index를 다른 형태로 나타낸듯?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Besides of being a plain old python object, '''torch_geometric.data.Data''' provides a number of utility functions, e.g.:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "['x', 'edge_index']\n"
     ]
    }
   ],
   "source": [
    "print(data.keys)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "tensor([[-1.],\n",
      "        [ 0.],\n",
      "        [ 1.]])\n"
     ]
    }
   ],
   "source": [
    "#노드의 피쳐들을 나타냄.\n",
    "print(data['x'])"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "edge_index found in data\n",
      "x found in data\n"
     ]
    }
   ],
   "source": [
    "for key, item in data:\n",
    "    print(\"{} found in data\".format(key))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 9,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "'edge_attr' in data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "3"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.num_nodes"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "4"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#단방향을 하나로 치고, 2*2 = 4\n",
    "data.num_edges"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1"
      ]
     },
     "execution_count": 12,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.num_node_features"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.contains_isolated_nodes()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.contains_self_loops()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "False"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#모두 양방향이니까\n",
    "data.is_directed()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 16,
   "metadata": {},
   "outputs": [],
   "source": [
    "device = torch.device('cuda')\n",
    "data = data.to(device)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can find a complete list of all methods at '''torch_geometric.data.Data'''\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Common Benchmark Datasets "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyTorch Geometric contains a large number of common benchmark datasets, e.g. all Planetoid datasets (Cora, Citeseer, Pubmed), all graph classification datasets from http://graphkernels.cs.tu-dortmund.de/ and their cleaned versions, the QM7 and QM9 dataset, and a handful of 3D mesh/point cloud datasets like FAUST, ModelNet10/40 and ShapeNet."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Initializing a dataset is straightforward. An initialization of a dataset will automatically download its raw files and process them to the previously described Data format. E.g., to load the ENZYMES dataset (consisting of 600 graphs within 6 classes), type:\n",
    "\n",
    "- raw file을 받아서 프로세싱 하는 것은 Data format칸에서 설명하고 여기에서는 간단한 것만 해보겠다."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch_geometric.datasets import TUDataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset = TUDataset(root='/tmp/ENZYMES', name='ENZYMES')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "600\n",
      "6\n",
      "3\n"
     ]
    }
   ],
   "source": [
    "print(len(dataset)) #600\n",
    "print(dataset.num_classes) #6\n",
    "print(dataset.num_node_features) #3"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Data(edge_index=[2, 168], x=[37, 3], y=[1])"
      ]
     },
     "execution_count": 10,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "#그래프 600개중에 첫 번째 그래프\n",
    "data = dataset[0]\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 11,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.is_undirected()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can see that the first graph in the dataset contains 37 nodes, each one having 3 features. There are 168/2 = 84 undirected edges and the graph is assigned to exactly one class. In addition, the data object is holding exactly one graph-level target.\n",
    "\n",
    "- 37개의 노드들이랑 3만큼의 피쳐를 가진다.\n",
    "- 168/2 만큼의 undirected edge를 가지고 \n",
    "- 그래프는 하나의 클래스를 가진다.\n",
    "- 여기서 학습시킬 것은 그래프 level의 target이다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can even use slices, long or byte tensors to split the dataset. E.g., to create a 90/10 train/test split, type:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "If you are unsure whether the dataset is already shuffled before you split, you can randomly permutate it by running:\n",
    "\n",
    "- 섞고싶을때 이걸 써라\n",
    "- 주석 친 두가지 모두 섞는 방법들임."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 22,
   "metadata": {},
   "outputs": [],
   "source": [
    "#dataset = dataset.shuffle() \n",
    "\n",
    "#perm = torch.randperm(len(dataset))\n",
    "#dataset = dataset[perm]\n",
    "\n",
    "train_dataset = dataset[:540] #ENZYMES(540)\n",
    "test_dataset = dataset[540:] #ENZYMES(60)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let’s try another one! Let’s download '''Cora''', the standard benchmark dataset for semi-supervised graph node classification:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch_geometric.datasets import Planetoid"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 13,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Cora()"
      ]
     },
     "execution_count": 13,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "dataset = Planetoid(root='/tmp/Cora', name='Cora')\n",
    "dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 14,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1433"
      ]
     },
     "execution_count": 14,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "len(dataset) #1\n",
    "dataset.num_classes #7\n",
    "dataset.num_node_features #1433"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Here, the dataset contains only a single, undirected citation graph:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 15,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Data(edge_index=[2, 10556], test_mask=[2708], train_mask=[2708], val_mask=[2708], x=[2708, 1433], y=[2708])"
      ]
     },
     "execution_count": 15,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data = dataset[0]\n",
    "data"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 33,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "True"
      ]
     },
     "execution_count": 33,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.is_undirected()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This time, the Data objects holds a label for each node, and additional attributes: train_mask, val_mask and test_mask:\n",
    "\n",
    "- train_mask\n",
    "    - denotes against which nodes to train (140 nodes)\n",
    "    \n",
    "-  val_mask\n",
    "    - denotes which nodes to use for validation, e.g., to perform early stopping (500 nodes)\n",
    "    \n",
    "- test_mask\n",
    "    - denotes against which nodes to test (1000 nodes)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 37,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "140"
      ]
     },
     "execution_count": 37,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.train_mask.sum().item()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 35,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "500"
      ]
     },
     "execution_count": 35,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.val_mask.sum().item()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 36,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "1000"
      ]
     },
     "execution_count": 36,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "data.test_mask.sum().item()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Mini-batches"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Neural networks are usually trained in a batch-wise fashion. PyTorch Geometric achieves parallelization over a mini-batch by creating sparse block diagonal adjacency matrices (defined by edge_index and edge_attr) and concatenating feature and target matrices in the node dimension. This composition allows differing number of nodes and edges over examples in one batch:\n",
    "\n",
    "- edge_index와 edge_attr을 통해 sparse한 block diagonal adjacency matrices를 만든다. \n",
    "- 이 구성은 노드와 엣지수가 다른 example들이 한 배치에 있는 것을 가능하게 해준다."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "PyTorch Geometric contains its own '''torch_geometric.data.DataLoader''', which already takes care of this concatenation process. Let’s learn about it in an example:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 38,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch_geometric.datasets import TUDataset\n",
    "from torch_geometric.data import DataLoader\n",
    "\n",
    "dataset = TUDataset(root='/tmp/ENZYMES', name='ENZYMES', use_node_attr=True)\n",
    "loader = DataLoader(dataset, batch_size=32, shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 41,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Batch(batch=[1042], edge_index=[2, 4038], x=[1042, 21], y=[32])\n",
      "Batch(batch=[1170], edge_index=[2, 4368], x=[1170, 21], y=[32])\n",
      "Batch(batch=[1027], edge_index=[2, 4070], x=[1027, 21], y=[32])\n",
      "Batch(batch=[977], edge_index=[2, 3790], x=[977, 21], y=[32])\n",
      "Batch(batch=[1058], edge_index=[2, 3994], x=[1058, 21], y=[32])\n",
      "Batch(batch=[1038], edge_index=[2, 4014], x=[1038, 21], y=[32])\n",
      "Batch(batch=[963], edge_index=[2, 3726], x=[963, 21], y=[32])\n",
      "Batch(batch=[916], edge_index=[2, 3630], x=[916, 21], y=[32])\n",
      "Batch(batch=[1084], edge_index=[2, 3702], x=[1084, 21], y=[32])\n",
      "Batch(batch=[1068], edge_index=[2, 3856], x=[1068, 21], y=[32])\n",
      "Batch(batch=[1013], edge_index=[2, 3862], x=[1013, 21], y=[32])\n",
      "Batch(batch=[1136], edge_index=[2, 4350], x=[1136, 21], y=[32])\n",
      "Batch(batch=[983], edge_index=[2, 3854], x=[983, 21], y=[32])\n",
      "Batch(batch=[1017], edge_index=[2, 3940], x=[1017, 21], y=[32])\n",
      "Batch(batch=[1139], edge_index=[2, 4346], x=[1139, 21], y=[32])\n",
      "Batch(batch=[1180], edge_index=[2, 4368], x=[1180, 21], y=[32])\n",
      "Batch(batch=[926], edge_index=[2, 3538], x=[926, 21], y=[32])\n",
      "Batch(batch=[1017], edge_index=[2, 3904], x=[1017, 21], y=[32])\n",
      "Batch(batch=[826], edge_index=[2, 3214], x=[826, 21], y=[24])\n"
     ]
    }
   ],
   "source": [
    "for batch in loader:\n",
    "    batch\n",
    "    print(batch)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "'''torch_geometric.data.Batch''' inherits from '''torch_geometric.data.Data''' and contains an additional attribute: '''batch'''.\n",
    "\n",
    "'''batch''' is a column vector which maps each node to its respective graph in the batch:"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can use it to, e.g., '''average node features''' in the node dimension for each graph individually:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 42,
   "metadata": {},
   "outputs": [],
   "source": [
    "from torch_scatter import scatter_mean\n",
    "from torch_geometric.datasets import TUDataset\n",
    "from torch_geometric.data import DataLoader\n",
    "\n",
    "dataset = TUDataset(root='/tmp/ENZYMES', name='ENZYMES', use_node_attr=True)\n",
    "loader = DataLoader(dataset, batch_size=32, shuffle=True)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 43,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "torch.Size([32, 21])\n",
      "torch.Size([32, 21])\n",
      "torch.Size([32, 21])\n",
      "torch.Size([32, 21])\n",
      "torch.Size([32, 21])\n",
      "torch.Size([32, 21])\n",
      "torch.Size([32, 21])\n",
      "torch.Size([32, 21])\n",
      "torch.Size([32, 21])\n",
      "torch.Size([32, 21])\n",
      "torch.Size([32, 21])\n",
      "torch.Size([32, 21])\n",
      "torch.Size([32, 21])\n",
      "torch.Size([32, 21])\n",
      "torch.Size([32, 21])\n",
      "torch.Size([32, 21])\n",
      "torch.Size([32, 21])\n",
      "torch.Size([32, 21])\n",
      "torch.Size([24, 21])\n"
     ]
    }
   ],
   "source": [
    "for data in loader:\n",
    "    x = scatter_mean(data.x, data.batch, dim=0)\n",
    "    print(x.size())\n",
    "    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Data Transforms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Transforms are a common way in '''torchvision''' to transform images and perform augmentation. PyTorch Geometric comes with its own transforms, which expect a '''Data''' object as input and return a new transformed '''Data''' object. Transforms can be chained together using '''torch_geometric.transforms.Compose''' and are applied before saving a processed dataset on disk ('''pre_transform''') or before accessing a graph in a dataset ('''transform''').\n",
    "\n",
    "- torchvision말고 별도의 transform이 가능하다.\n",
    "- Data obj --> Data obj\n",
    "- pre_transform으로 지정\n",
    "- torch_geometric.transforms"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let’s look at an example, where we apply transforms on the ShapeNet dataset (containing 17,000 3D shape point clouds and per point labels from 16 shape categories)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 47,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Data(category=[1], pos=[2518, 3], x=[2518, 3], y=[2518])"
      ]
     },
     "execution_count": 47,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from torch_geometric.datasets import ShapeNet\n",
    "\n",
    "dataset = ShapeNet(root='/tmp/ShapeNet', categories=['Airplane'])\n",
    "\n",
    "dataset[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We can convert the point cloud dataset into a graph dataset by generating nearest neighbor graphs from the point clouds via transforms:\n",
    "\n",
    "- point cloud dataset --> a graph dataset(KNN graphs viia transforms)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 52,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Data(category=[1], pos=[2518, 3], x=[2518, 3], y=[2518])"
      ]
     },
     "execution_count": 52,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch_geometric.transforms as T\n",
    "from torch_geometric.datasets import ShapeNet\n",
    "\n",
    "dataset = ShapeNet(root='/tmp/ShapeNet', categories=['Airplane'],\n",
    "                    pre_transform=T.KNNGraph(k=6))\n",
    "\n",
    "dataset[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Note\n",
    "\n",
    "We use the '''pre_transform''' to convert the data before saving it to disk (leading to faster loading times). Note that the next time the dataset is initialized it will already contain graph edges, even if you do not pass any transform."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "In addition, we can use the transform argument to '''randomly augment''' a Data object, e.g. translating each node position by a small number:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 53,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Data(category=[1], pos=[2518, 3], x=[2518, 3], y=[2518])"
      ]
     },
     "execution_count": 53,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "import torch_geometric.transforms as T\n",
    "from torch_geometric.datasets import ShapeNet\n",
    "\n",
    "dataset = ShapeNet(root='/tmp/ShapeNet', categories=['Airplane'],\n",
    "                    pre_transform=T.KNNGraph(k=6),\n",
    "                    transform=T.RandomTranslate(0.01))\n",
    "\n",
    "dataset[0]"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "You can find a complete list of all implemented transforms at '''torch_geometric.transforms'''."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Learning Methods on Graphs\n",
    "\n",
    "After learning about data handling, datasets, loader and transforms in PyTorch Geometric, it’s time to implement our first graph neural network!\n",
    "\n",
    "We will use a simple GCN layer and replicate the experiments on the Cora citation dataset. For a high-level explanation on GCN, have a look at its blog post.\n",
    "\n",
    "We first need to load the Cora dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 55,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "text/plain": [
       "Cora()"
      ]
     },
     "execution_count": 55,
     "metadata": {},
     "output_type": "execute_result"
    }
   ],
   "source": [
    "from torch_geometric.datasets import Planetoid\n",
    "\n",
    "dataset = Planetoid(root='/tmp/Cora', name='Cora')\n",
    "dataset"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that we do not need to use transforms or a dataloader. Now let’s implement a two-layer GCN:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 56,
   "metadata": {},
   "outputs": [],
   "source": [
    "import torch\n",
    "import torch.nn.functional as F\n",
    "from torch_geometric.nn import GCNConv\n",
    "\n",
    "class Net(torch.nn.Module):\n",
    "    def __init__(self):\n",
    "        super(Net, self).__init__()\n",
    "        self.conv1 = GCNConv(dataset.num_node_features, 16)\n",
    "        self.conv2 = GCNConv(16, dataset.num_classes)\n",
    "\n",
    "    def forward(self, data):\n",
    "        x, edge_index = data.x, data.edge_index\n",
    "\n",
    "        x = self.conv1(x, edge_index)\n",
    "        x = F.relu(x)\n",
    "        x = F.dropout(x, training=self.training)\n",
    "        x = self.conv2(x, edge_index)\n",
    "\n",
    "        return F.log_softmax(x, dim=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The constructor defines two '''GCNConv''' layers which get called in the forward pass of our network. Note that the non-linearity is not integrated in the conv calls and hence needs to be applied afterwards (something which is consistent accross all operators in PyTorch Geometric). Here, we chose to use ReLU as our intermediate non-linearity between and finally output a softmax distribution over the number of classes. Let’s train this model on the train nodes for 200 epochs:\n",
    "\n",
    "- forward path로 2 layer의 GCNCConv를 만들었다. \n",
    "- conv에는 비선형성이 존재하지않음으로 따로 만들어주었다. \n",
    "- ReLU, softmax를 선택하였다. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 57,
   "metadata": {},
   "outputs": [],
   "source": [
    "device = torch.device('cuda' if torch.cuda.is_available() else 'cpu')\n",
    "model = Net().to(device)\n",
    "data = dataset[0].to(device)\n",
    "optimizer = torch.optim.Adam(model.parameters(), lr=0.01, weight_decay=5e-4)\n",
    "\n",
    "model.train()\n",
    "for epoch in range(200):\n",
    "    optimizer.zero_grad()\n",
    "    out = model(data)\n",
    "    loss = F.nll_loss(out[data.train_mask], data.y[data.train_mask])\n",
    "    loss.backward()\n",
    "    optimizer.step()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 58,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Accuracy: 0.8070\n"
     ]
    }
   ],
   "source": [
    "model.eval()\n",
    "_, pred = model(data).max(dim=1)\n",
    "correct = float (pred[data.test_mask].eq(data.y[data.test_mask]).sum().item())\n",
    "acc = correct / data.test_mask.sum().item()\n",
    "print('Accuracy: {:.4f}'.format(acc))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "That is all it takes to implement your first graph neural network. The easiest way to learn more about graph convolution and pooling is to study the examples in the '''examples/''' directory and to browse '''torch_geometric.nn'''. Happy hacking!"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.4"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}