Introduction

This repository contains the source code for the publications GRAND: Graph Neural Diffusion and Beltrami Flow and Neural Diffusion on Graphs (BLEND). These approaches treat deep learning on graphs as a continuous diffusion process and Graph Neural Networks (GNNs) as discretisations of an underlying PDE. In both models, the layer structure and topology correspond to the discretisation choices of temporal and spatial operators. Our approach allows a principled development of a broad new class of GNNs that are able to address the common plights of graph learning models such as depth, oversmoothing, and bottlenecks. Key to the success of our models are stability with respect to perturbations in the data and this is addressed for both implicit and explicit discretisation schemes. We develop linear and nonlinear versions of GRAND, which achieve competitive results on many standard graph benchmarks. BLEND is a non-Euclidean extension of GRAND that jointly evolves the feature and positional encodings of each node providing a principled means to perform graph rewiring.

Running the experiments

Requirements

Dependencies (with python >= 3.7): Main dependencies are torch==1.8.1 torch-cluster==1.5.9 torch-geometric==1.7.0 torch-scatter==2.0.6 torch-sparse==0.6.9 torch-spline-conv==1.2.1 torchdiffeq==0.2.1 Commands to install all the dependencies in a new conda environment

conda create --name grand python=3.7
conda activate grand

pip install ogb pykeops
pip install torch==1.8.1
pip install torchdiffeq -f https://pytorch-geometric.com/whl/torch-1.8.1+cu102.html

pip install torch-scatter -f https://pytorch-geometric.com/whl/torch-1.8.1+cu102.html
pip install torch-sparse -f https://pytorch-geometric.com/whl/torch-1.8.1+cu102.html
pip install torch-cluster -f https://pytorch-geometric.com/whl/torch-1.8.1+cu102.html
pip install torch-spline-conv -f https://pytorch-geometric.com/whl/torch-1.8.1+cu102.html
pip install torch-geometric

Troubleshooting

There is a bug in pandas==1.3.1 that could produce the error ImportError: cannot import name 'DtypeObj' from 'pandas._typing' If encountered, then the fix is pip install pandas==1.3.0 -U

GRAND (Graph Neural Diffusion)

Dataset and Preprocessing

create a root level folder

./data

This will be automatically populated the first time each experiment is run.

Experiments

For example to run for Cora with random splits:

cd src
python run_GNN.py --dataset Cora 

BLEND (Beltrami Flow and Neural Diffusion on Graphs)

Dataset and Preprocessing

Create a root level

./data folder

This will be automatically populated the first time each experiment is run. create a root level folder

./data/pos_encodings

DIGL positional encodings will build automatically and populate this folder, but DeepWalk or Hyperbollic positional encodings will need generating using the relevant generator scripts or downloading. We include a shell script (warning: it's slow) to generate them:

sh deepwalk_gen.sh

then create symlinks to them with

python deepwalk_gen_symlinks.py

Alternatively, we also provide precomputed positional encodings here Specifically, the positional encodings required to run the default settings for Citeseer, Computers, Phota and ogbn-arxiv are

• Citeseer
• Computers
• Photo
• ogbn-arxiv

Download them and place into

./data/pos_encodings

Experiments

For example to run for Cora with random splits:

cd src
python run_GNN.py --dataset Cora --beltrami

Troubleshooting

Most problems installing the dependencies are caused by Cuda version mismatches with pytorch geometric. We recommend checking your cuda and pytorch versions

nvcc --version
python -c "import torch; print(torch.__version__)"

and then following instructions here to install pytorch geometric https://pytorch-geometric.readthedocs.io/en/latest/notes/installation.html

Cite us

If you found this work useful, please consider citing our papers

@article
{chamberlain2021grand,
  title={GRAND: Graph Neural Diffusion},
  author={Chamberlain, Benjamin Paul and Rowbottom, James and Goronova, Maria and Webb, Stefan and Rossi, 
  Emanuele and Bronstein, Michael M},
  journal={Proceedings of the 38th International Conference on Machine Learning,
               (ICML) 2021, 18-24 July 2021, Virtual Event},
  year={2021}
}

and

@article
{chamberlain2021blend,
  title={Beltrami Flow and Neural Diffusion on Graphs},
  author={Chamberlain, Benjamin Paul and Rowbottom, James and Eynard, Davide and Di Giovanni, Francesco and Dong Xiaowen and Bronstein, Michael M},
  journal={Proceedings of the Thirty-fifth Conference on Neural Information Processing Systems (NeurIPS) 2021, Virtual Event},
  year={2021}
}

Security Issues?

Please report sensitive security issues via Twitter's bug-bounty program (https://hackerone.com/twitter) rather than GitHub.