# Graph Machine Learning in the Era of Large Language Models (LLMs)

**Authors:** Wenqi Fan, Shijie Wang, Jiani Huang, Zhikai Chen, Yu Song, Wenzhuo Tang, Haitao Mao, Hui Liu, Xiaorui Liu, Dawei Yin, Qing Li  
**arXiv:** [2404.14928](https://arxiv.org/abs/2404.14928)  
**Date:** 2024-04-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2404.14928v2/

## Key points

- Graphs are essential for representing complex relationships in machine learning across domains like social networks, knowledge graphs, and molecular discovery.
- Graph Neural Networks (GNNs) have become a fundamental component in Graph Machine Learning (Graph ML) due to the emergence of deep learning, enabling effective representation and processing of graph structures.
- Large Language Models (LLMs) have demonstrated exceptional capabilities in language-related tasks and have been integrated into various applications like computer vision and recommender systems.
- Interest in applying LLMs within the graph domain has grown, with exploration focused on enhancing generalization, transferability, and few-shot learning abilities in Graph ML.
- Leveraging knowledge graphs can enhance the reasoning capabilities of LLMs and potentially address limitations such as hallucinations and explainability gaps.
- There is a need for a comprehensive review to capture the latest advancements in Graph ML amidst the era of LLMs, exploring how LLMs can improve graph feature quality, reduce reliance on labeled data, and address challenges like graph heterogeneity and out-of-distribution generalization.
- The fusion of LLMs with graphs shows promise across various applications while also opening up potential future directions for research in this field.

## Summary

In the realm of machine learning, graphs serve as a crucial tool for representing intricate relationships across various domains such as social networks, knowledge graphs, and molecular discovery. The emergence of deep learning has paved the way for Graph Neural Networks (GNNs) to become a fundamental component in Graph Machine Learning (Graph ML), enabling the effective representation and processing of graph structures. Concurrently, Large Language Models (LLMs) have showcased unparalleled capabilities in language-related tasks and have been widely integrated into diverse applications like computer vision and recommender systems. The remarkable success of LLMs has sparked interest in their application within the graph domain, leading to extensive exploration of their potential in enhancing Graph ML's generalization, transferability, and few-shot learning abilities. Notably, knowledge graphs are particularly rich sources of reliable factual knowledge that can be leveraged to bolster the reasoning capabilities of LLMs and potentially mitigate their limitations such as hallucinations and explainability gaps. Given the rapid advancements in this research direction, there is a pressing need for a comprehensive review that encapsulates the latest progressions in Graph ML amidst the era of LLMs. This survey delves into recent developments within Graph ML before delving into how LLMs can be harnessed to enhance graph feature quality, reduce reliance on labeled data, and tackle challenges like graph heterogeneity and out-of-distribution (OOD) generalization. Furthermore, it explores how graphs can augment LLM performance by enhancing pre-training processes and inference mechanisms. Moreover, this survey investigates various applications where the fusion of LLMs with graphs holds promise while also discussing potential future directions for this burgeoning field. Authored by Wenqi Fan et al., this study provides valuable insights for researchers and practitioners seeking an in-depth understanding of how LLMs are reshaping Graph ML methodologies and applications.