# Probing Graph Representations

**Authors:** Mohammad Sadegh Akhondzadeh, Vijay Lingam, Aleksandar Bojchevski  
**arXiv:** [2303.03951](https://arxiv.org/abs/2303.03951)  
**Date:** 2023-03-07  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.03951v1/

## Key points

- The paper addresses the question of what is encoded in the learned representations of Graph Neural Networks (GNNs)
- The authors propose a probing framework to quantify meaningful information in graph representations
- The probing framework is applied to molecular datasets and shows potential for understanding inductive biases of graph-based models
- Transformer-based models capture more chemically relevant information compared to message passing models
- Design choices such as skip connections and virtual nodes are investigated for their impact on learned representations
- Probing is advocated as a valuable diagnostic tool for evaluating graph-based models
- Insights into the information encoded by GNNs and how different model architectures and design choices affect performance can be gained using this framework
- The study contributes to our understanding of GNNs by shedding light on the content encoded within their learned representations
- Both theoretical knowledge and empirical evaluations are important when developing and assessing graph-based models.

## Summary

In their paper titled "Probing Graph Representations," Mohammad Sadegh Akhondzadeh, Vijay Lingam, and Aleksandar Bojchevski address the question of what is encoded in the learned representations of Graph Neural Networks (GNNs). To tackle this question, the authors propose a probing framework that quantifies the meaningful information contained in graph representations. The authors apply their probing framework to molecular datasets and demonstrate its potential for understanding the inductive biases of graph-based models. They compare different families of models and find that transformer-based models capture more chemically relevant information compared to models based on message passing. Additionally, they investigate the impact of design choices such as skip connections and virtual nodes on the learned representations. Through their experiments and analyses, Akhondzadeh et al. advocate for probing as a valuable diagnostic tool for evaluating graph-based models. By using this framework, researchers can gain insights into what kind of information GNNs encode and how different model architectures and design choices affect their performance. Overall, this study contributes to our understanding of GNNs by shedding light on the content encoded within their learned representations. The findings highlight the importance of considering both theoretical knowledge and empirical evaluations when developing and assessing graph-based models.