Graph Neural PDEs

Description

Graph Neural PDEs is a GitHub repository developed by twitter-research, offering the source code for their publications GRAND (Graph Neural Diffusion) and BLEND (Beltrami Flow and Neural Diffusion on Graphs). These models conceptualize deep learning on graphs as a continuous diffusion process, with Graph Neural Networks (GNNs) serving as discretizations of underlying Partial Differential Equations (PDEs).

The core innovation lies in how the layer structure and topology of GNNs are mapped to the discretization choices of temporal and spatial operators. This principled development leads to a new class of GNNs designed to overcome prevalent issues in graph learning, such as depth limitations, oversmoothing, and bottlenecks. A key aspect of these models is their stability with respect to data perturbations, addressed through both implicit and explicit discretization schemes.

The repository includes implementations for linear and nonlinear versions of GRAND, which have demonstrated competitive performance on various standard graph benchmarks. BLEND extends GRAND by operating in a non-Euclidean space, jointly evolving node feature and positional encodings. This provides a robust method for graph rewiring. The project also details the requirements and commands for setting up the necessary dependencies, primarily focusing on PyTorch and its related libraries, and offers troubleshooting advice for common installation issues, particularly those related to CUDA version mismatches.

For users interested in running the experiments, the repository outlines dataset and preprocessing steps for both GRAND and BLEND. It specifies the creation of a `./data` folder for datasets and, for BLEND, an additional `./data/pos_encodings` folder for positional encodings. Instructions are provided for running experiments on datasets like Cora, Citeseer, Computers, Photo, and ogbn-arxiv, along with guidance on generating or downloading positional encodings.

The project encourages contributions and reports security issues through Twitter's bug-bounty program. It is licensed under the Apache-2.0 license, making it freely available for use and modification. The repository also includes citations for the GRAND and BLEND papers, allowing researchers to reference the foundational work.

Graph Neural PDEs Highlights

• Implementation of GRAND (Graph Neural Diffusion) model

• Implementation of BLEND (Beltrami Flow and Neural Diffusion on Graphs) model

• Treats deep learning on graphs as a continuous diffusion process

• Models GNNs as discretizations of underlying PDEs

• Addresses graph learning challenges like oversmoothing and depth limitations

• Provides stability with respect to data perturbations

• Includes linear and nonlinear versions of GRAND

• BLEND offers non-Euclidean extension for graph rewiring

• Detailed installation instructions for dependencies (PyTorch, etc.)

• Guidance on dataset and positional encoding preparation

• Scripts for running experiments on benchmark datasets

• Troubleshooting for dependency installation issues

• Apache-2.0 License

Getting Started with Graph Neural PDEs

1. 1

   Set up environment: Create and activate a conda environment with specified Python version.

2. 2

   Install dependencies: Use pip to install required libraries including PyTorch, PyTorch Geometric, and others.

3. 3

   Prepare data: Create a `./data` folder for datasets and `./data/pos_encodings` for BLEND.

4. 4

   Generate positional encodings: Use provided scripts or download precomputed encodings if needed for BLEND.

5. 5

   Run experiments: Navigate to the `src` directory and execute `run_GNN.py` with dataset parameters.

6. 6

   Troubleshoot installation: Verify CUDA and PyTorch versions if encountering installation errors.

Graph Neural PDEs's Use Cases

• Graph Neural Network Research
• Node Classification
• Graph Rewiring
• Addressing Oversmoothing
• Benchmarking GNN Models
• Continuous Graph Learning

FAQ from Graph Neural PDEs

• What is Graph Neural PDEs?

  Graph Neural PDEs is an open-source project on GitHub that provides code for Graph Neural Networks (GNNs) based on diffusion processes and partial differential equations, specifically implementing the GRAND and BLEND models.

• What problems does Graph Neural PDEs aim to solve?

  It aims to address common graph learning issues such as depth limitations, oversmoothing, and bottlenecks by treating graph learning as a continuous diffusion process.

• What are the main models implemented in this repository?

  The repository implements GRAND (Graph Neural Diffusion) and BLEND (Beltrami Flow and Neural Diffusion on Graphs).

• How does BLEND differ from GRAND?

  BLEND is a non-Euclidean extension of GRAND that jointly evolves node feature and positional encodings, providing a method for graph rewiring.

• What are the key dependencies for running Graph Neural PDEs?

  Key dependencies include PyTorch (torch), torch-cluster, torch-geometric, torch-scatter, torch-sparse, torch-spline-conv, and torchdiffeq, along with python >= 3.7.

• How can I install the required dependencies?

  You can install dependencies by creating a conda environment and using pip commands as detailed in the repository's README file.

• What should I do if I encounter installation issues?

  Troubleshooting often involves checking for CUDA version mismatches with PyTorch Geometric and referring to the official PyTorch Geometric installation guide.

• Where can I find the datasets and positional encodings?

  Datasets are typically populated automatically in a `./data` folder when experiments are run. Positional encodings for BLEND may need to be generated or downloaded into `./data/pos_encodings`.

• How do I run an experiment, for example, on the Cora dataset?

  Navigate to the `src` directory and run the command `python run_GNN.py --dataset Cora`.

• What is the license for this project?

  The project is licensed under the Apache-2.0 license.

• How can I cite the work presented in this repository?

  You can cite the GRAND and BLEND papers, with full citation details provided in the README file.