# Towards Efficient Generative Large Language Model Serving: A Survey from Algorithms to Systems

**Authors:** Xupeng Miao, Gabriele Oliaro, Zhihao Zhang, Xinhao Cheng, Hongyi Jin, Tianqi Chen, Zhihao Jia  
**arXiv:** [2312.15234](https://arxiv.org/abs/2312.15234)  
**Date:** 2023-12-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2312.15234v1/

## Key points

- Generative Large Language Models (LLMs) are driving advancements in language-related tasks such as machine translation, sentiment analysis, question answering, and text generation.
- Transformer-based architectures like the GPT-family have revolutionized Natural Language Processing (NLP) tasks and expanded applications to automated programming, science discovery, personalized digital assistants, creative arts, and next-generation computing architecture.
- Challenges exist with LLMs due to their computational requirements during serving, including concerns over energy consumption, scalability, and accessibility for broader adoption.
- The survey paper explores efficient LLM serving methodologies proposed by the research community to address these challenges.
- Solutions range from algorithmic innovations to novel system architectures aimed at optimizing the inference process for large language models.
- Cutting-edge modifications and groundbreaking changes in system designs aim to offer valuable insights for researchers and practitioners seeking effective LLM deployment.

## Summary

The rapidly evolving landscape of Artificial Intelligence (AI) has seen the emergence of generative Large Language Models (LLMs) as a driving force behind significant advancements in various language-related tasks. These models have showcased exceptional performance and versatility in areas such as machine translation, sentiment analysis, question answering, and text generation. The advent of Transformer-based architectures, including the GPT-family and other latest public LLMs, has revolutionized Natural Language Processing (NLP) tasks and expanded their application to automated programming, science discovery, personalized digital assistants, creative arts, and next-generation computing architecture. However, the success of LLMs has also brought forth challenges related to their computational requirements during serving. The substantial model size and complexity necessitate extensive computational resources for deployment in real-world applications. This resource-intensive nature raises concerns over energy consumption, scalability, and accessibility for broader adoption beyond large companies with rich compute resources. This survey paper addresses the critical need for efficient LLM serving methodologies by exploring a range of strategies proposed by the research community. It provides an in-depth analysis of solutions spanning from algorithmic innovations to novel system architectures aimed at optimizing the inference process for large language models. By delving into cutting-edge modifications and groundbreaking changes in system designs, this survey aims to offer valuable insights for researchers and practitioners seeking to overcome barriers in effective LLM deployment. Ultimately,this comprehensive understanding of current state-of-the-art practices and future directions in efficient LLM serving is poised to reshape the future of AI innovation.