# Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs

**Authors:** Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu, Jiliang Tang  
**arXiv:** [2307.03393](https://arxiv.org/abs/2307.03393)  
**Date:** 2023-07-07  
**Summary page:** https://www.summarizepaper.com/en/paper/2307.03393v4/

## Key points

- Learning on Graphs is a crucial area of study in machine learning with wide real-world applications.
- Graph Neural Networks (GNNs) are commonly used for learning on graphs, often with shallow text embeddings for node attributes.
- Large Language Models (LLMs) have extensive common knowledge and robust semantic comprehension abilities, revolutionizing text data handling workflows.
- Researchers led by Zhikai Chen et al. explored integrating LLMs into graph machine learning for node classification tasks.
- Two pipelines were investigated: LLMs-as-Enhancers and LLMs-as-Predictors, showing promising results in enhancing nodes' attributes and standalone prediction capabilities.
- The research team conducted comprehensive studies under various scenarios, uncovering valuable insights for future research directions.
- All codes and datasets related to the study are openly available at https://github.com/CurryTang/Graph-LLM.
- The exploration titled "Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs" will soon appear in SIGKDD Explorations journal.

## Summary

In the realm of machine learning, Learning on Graphs has emerged as a crucial area of study due to its wide array of real-world applications. One of the predominant methods for learning on graphs involves utilizing Graph Neural Networks (GNNs) in conjunction with textual node attributes. However, this approach often relies on shallow text embeddings as initial node representations, which can be limiting in terms of general knowledge and deep semantic understanding. In recent years, there has been a significant shift towards leveraging Large Language Models (LLMs) in various machine learning tasks. These LLMs have demonstrated an impressive capacity for possessing extensive common knowledge and robust semantic comprehension abilities, thereby revolutionizing existing workflows for handling text data. Building upon this advancement, a group of researchers led by Zhikai Chen, Haitao Mao, Hang Li, Wei Jin, Hongzhi Wen, Xiaochi Wei, Shuaiqiang Wang, Dawei Yin, Wenqi Fan, Hui Liu and Jiliang Tang set out to explore the potential of integrating LLMs into graph machine learning. Specifically focusing on the node classification task within graph structures,the researchers investigated two distinct pipelines: LLMs-as-Enhancers and LLMs-as-Predictors. The former approach involves leveraging LLMs to enhance nodes' text attributes by tapping into their vast knowledge base before generating predictions through GNNs. On the other hand,the latter pipeline explores using LLMs as standalone predictors without relying on additional models. Through comprehensive and systematic studies conducted under various settings and scenarios,the research team uncovered valuable insights and made original observations regarding the efficacy of integrating LLMs into graph machine learning workflows. Their findings not only shed light on new possibilities but also suggest promising directions for future research endeavors in this domain. It is worth noting that all codes and datasets related to this study are openly available at https://github.com/CurryTang/Graph-LLM. This comprehensive exploration titled "Exploring the Potential of Large Language Models (LLMs) in Learning on Graphs" is slated to appear in SIGKDD Explorations journal soon.