# LLMParser: A LLM-based Log Parsing Framework

**Authors:** Zhihan Jiang, Jinyang Liu, Zhuangbin Chen, Yichen Li, Junjie Huang, Yintong Huo, Pinjia He, Jiazhen Gu, Michael R. Lyu  
**arXiv:** [2310.01796](https://arxiv.org/abs/2310.01796)  
**Date:** 2023-10-03  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2310.01796v1/

## Key points

- Introduction of LLMParser as a novel log parsing framework based on large language models (LLMs)
- Utilization of in-context learning (ICL) capability and hierarchical candidate sampling algorithm for accuracy and robustness
- Adaptive parsing cache component to store and refine log templates generated by the LLM for efficiency
- Extensive evaluation showing LLMParser outperforming state-of-the-art methods in accuracy and efficiency
- Comparable efficiency to the most efficient baseline method, Drain, by reducing query times to LLMs

## Summary

The paper "LLMParser: A LLM-based Log Parsing Framework" introduces a novel approach to log parsing using large language models (LLMs). Log parsing is essential for various log analysis tasks and has traditionally relied on human-made rules or limited training data. However, with the rise of powerful LLMs, there is an opportunity for more accurate log parsing by leveraging their pre-trained knowledge related to code and logging. To address this potential, the authors propose LLMParser - the first practical LLM-based log parsing framework. LLMParser enhances accuracy and robustness by utilizing the in-context learning (ICL) capability of LLMs and employing a hierarchical candidate sampling algorithm while selecting high-quality demonstrations. Additionally, it features an adaptive parsing cache component that stores and refines templates generated by the LLM. This design improves efficiency by quickly matching previously parsed log templates and ensuring consistent results through adaptive template updates. Extensive evaluation on large-scale public datasets demonstrates that LLMParser outperforms state-of-the-art methods in terms of accuracy and efficiency. By reducing query times to LLMs, it achieves comparable efficiency to the most efficient baseline method known as Drain. The collaborative effort of authors Zhihan Jiang, Jinyang Liu, Zhuangbin Chen, Yichen Li, Junjie Huang, Yintong Huo, Pinjia He, Jiazhen Gu, and Michael R. Lyu showcases the potential of LLM-based approaches in advancing log parsing techniques for improved log analysis tasks.