# Knowledge Distillation on Graphs: A Survey

**Authors:** Yijun Tian, Shichao Pei, Xiangliang Zhang, Chuxu Zhang, Nitesh V. Chawla  
**arXiv:** [2302.00219](https://arxiv.org/abs/2302.00219)  
**Date:** 2023-02-01  
**Summary page:** https://www.summarizepaper.com/en/paper/2302.00219v1/

## Key points

- Graph Neural Networks (GNNs) have gained interest for handling graph data effectively.
- Deploying GNNs on resource-limited devices faces challenges due to model sizes and scalability issues.
- Knowledge distillation on graphs (KDG) aims to create a more compact and efficient model for enhanced performance.
- Recent advancements in KDG have shown promising results with numerous studies conducted.
- A comprehensive survey by Yijun Tian et al. reviews existing works on KDG, addressing key questions and categorizing diverse approaches.
- The survey provides insights into the progress made in KDG research and offers perspectives for future exploration.

## Summary

In recent years, there has been a surge of interest in the ability of Graph Neural Networks (GNNs) to effectively handle graph data. However, deploying GNNs on resource-limited devices poses a significant challenge due to constraints related to model sizes and scalability issues stemming from multi-hop data dependencies. Real-world graphs often exhibit intricate structural information and features, further complicating the utilization of GNNs. To address these challenges and enhance the practicality of GNNs by fully capturing complex topological information, the concept of knowledge distillation on graphs (KDG) has emerged. KDG aims to construct a more compact yet efficient model that can extract additional insights from data, facilitating model compression and performance enhancement. Recent advancements in KDG have shown promising results with numerous studies being conducted in this domain. In a comprehensive survey conducted by Yijun Tian, Shichao Pei, Xiangliang Zhang, Chuxu Zhang, and Nitesh V. Chawla , existing works on KDG are systematically reviewed. The survey delves into various aspects of KDG by addressing key questions such as what information should be distilled , who should perform the distillation process , and how the distillation should be carried out . By categorizing and summarizing the diverse approaches taken in KDG research , this survey sheds light on the progress made in this field and provides valuable insights for future exploration. The authors not only outline the challenges and foundations of KDG but also offer perspectives on potential directions for further research endeavors. Through this detailed examination of knowledge distillation on graphs, researchers can gain a deeper understanding of how to leverage this technique for optimizing GNN models and extracting richer knowledge from graph data sets.