Simplifying Graph Convolutional Networks

Description

This GitHub repository hosts the official implementation of the Simplifying Graph Convolutional Networks (SGC) model, as detailed in the ICML2019 paper. SGC is an advancement in Graph Convolutional Networks (GCNs) that simplifies the architecture by removing nonlinearities and collapsing weight matrices. This transformation results in a linear model that retains competitive performance while drastically reducing training time and computational resources.

The implementation supports various graph datasets, including citation networks like Cora, Citeseer, and Pubmed, as well as social networks such as Reddit. For citation networks, the model achieves notable accuracy, for instance, 81.0% on Cora, 71.9% on Citeseer, and 78.9% on Pubmed, with training times as low as 0.13 seconds on a GTX 1080 Ti. For the Reddit dataset, it achieves an F1 score of 94.9% in 2.7 seconds.

The repository includes code for data preprocessing, model definition, training, and evaluation. It also provides guidance on hyperparameter tuning, particularly for weight decay, using tools like hyperopt. Practical advice is offered, such as normalizing features to have zero mean and standard deviation one to accelerate convergence, a technique beneficial for SGC and other linear models.

Beyond the core implementation, the repository points to additional resources and reference implementations in popular libraries like Deep Graph Library (DGL), PyTorch Geometric, Spektral, StellarGraph, and tf_geometric. It also features a work-in-progress branch for ablation studies and code for downstream tasks, starting with text classification. This project is a valuable resource for researchers and practitioners looking to leverage efficient and effective graph neural network models.

Simplifying Graph Convolutional Networks Highlights

• Official implementation of the Simplifying Graph Convolutional Networks (SGC) paper.

• Streamlined Graph Convolutional Network (GCN) architecture.

• Removes nonlinearities and collapses weight matrices for a linear model.

• Significantly reduced training time compared to traditional GCNs.

• Achieves competitive performance on benchmark datasets.

• Supports citation networks (Cora, Citeseer, Pubmed) and social networks (Reddit).

• Includes code for data preprocessing, model training, and evaluation.

• Provides guidance on hyperparameter tuning.

• Offers practical advice on feature normalization for accelerated convergence.

• Codebase for downstream tasks, including text classification.

• References to implementations in popular GNN libraries.

Getting Started with Simplifying Graph Convolutional Networks

1. 1

   Access model: Clone the GitHub repository.

2. 2

   Set up environment: Install dependencies using `pip install -r requirements.txt`.

3. 3

   Prepare data: Download the Reddit dataset and place it in the `data/` directory.

4. 4

   Train for citation networks: Run `python citation.py --dataset [dataset_name] --tuned`.

5. 5

   Train for Reddit: Run `python reddit.py --inductive --test`.

6. 6

   Integrate: Adapt the provided scripts for custom datasets or downstream tasks.

Simplifying Graph Convolutional Networks's Use Cases

• Node Classification
• Text Classification
• Graph Feature Learning
• Efficient GNN Training
• Benchmarking GNNs

FAQ from Simplifying Graph Convolutional Networks

• What is the main advantage of SGC over traditional GCNs?

  SGC simplifies the GCN architecture by removing nonlinearities and collapsing weight matrices, resulting in a linear model that trains significantly faster while maintaining competitive performance.

• Which datasets are supported by this implementation?

  This implementation supports citation networks (Cora, Citeseer, Pubmed) and the Reddit social network dataset.

• How can I install the dependencies for this project?

  You can install the required dependencies by running `pip install -r requirements.txt` in your environment.

• Is there a specific recommendation for accelerating SGC convergence?

  Yes, the repository suggests normalizing features to have zero mean with a standard deviation of one, which can accelerate convergence for SGC and other linear models.

• Where can I find implementations of SGC in other libraries?

  The repository lists reference implementations in Deep Graph Library, PyTorch Geometric, Spektral, StellarGraph, and tf_geometric.

• Does this repository include code for tasks beyond node classification?

  Yes, there is a `downstream` directory containing code for downstream tasks, starting with SGC implementation for text classification.

• What are the performance metrics and training times reported for SGC?

  For citation networks, SGC achieves accuracies like 81.0% on Cora. For Reddit, it achieves an F1 score of 94.9%. Training times are notably fast, often under a second for citation networks on a GTX 1080 Ti.

• How is hyperparameter tuning handled in this implementation?

  The repository mentions using hyperopt for tuning the weight decay hyperparameter, with results stored under `SGC-tuning`.

• What is the license for this repository?

  The repository is released under the MIT license.

• Can I use this implementation for inductive learning on the Reddit dataset?

  Yes, the usage instructions include running `python reddit.py --inductive --test` for inductive learning on the Reddit dataset.