# Text Summarization Using Large Language Models: A Comparative Study of MPT-7b-instruct, Falcon-7b-instruct, and OpenAI Chat-GPT Models

**Authors:** Lochan Basyal, Mihir Sanghvi  
**arXiv:** [2310.10449](https://arxiv.org/abs/2310.10449)  
**Date:** 2023-10-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.10449v1/

## Key points

- The paper explores text summarization with Large Language Models (LLMs), focusing on their capabilities and limitations.
- Various LLMs are investigated, and different hyperparameters are experimented with to evaluate the quality of generated summaries using metrics like BLEU Score, Rouge Score, and Bert Score.
- Text summarization methods are categorized into abstractive (rephrasing content) and extractive (selecting important sentences/phrases) approaches.
- Supervised summarization relies on labeled training data, while unsupervised summarization extracts information based on factors like sentence importance and coherence.
- Performance comparisons of LLMs such as MPT-7b-instruct, falcon-7b-instruct, and OpenAI ChatGPT text-davinci-003 models show that text-davinci-003 outperformed others in experiments on datasets like CNN Daily Mail and XSum.
- The research provides valuable insights for leveraging LLMs in NLP applications and lays the groundwork for advanced Generative AI solutions to address diverse business challenges.

## Summary

This paper delves into the realm of text summarization with Large Language Models (LLMs), providing a comprehensive exploration of their capabilities and limitations. The study investigates various LLMs and experiments with different hyperparameters to evaluate the quality of generated summaries using established metrics like BLEU Score, Rouge Score, and Bert Score. The primary focus is to offer valuable insights for leveraging LLMs in NLP applications and laying the groundwork for advanced Generative AI solutions that can address diverse business challenges. The paper is structured to include detailed explanations of text summarization methods, supervised and unsupervised techniques, datasets, evaluation metrics, inference with different LLMs, and suggestions for future enhancements. Text summarization methods are categorized into abstractive and extractive approaches. Abstractive summarization involves generating concise summaries by understanding context and rephrasing content using advanced language models like LLMs. On the other hand, extractive summarization selects important sentences or phrases directly from the source text without rephrasing. Supervised summarization relies on labeled training data where human annotators provide summaries for source texts. Machine learning models are trained on this data to learn mappings between texts and summaries. Unsupervised summarization does not require labeled data; it extracts relevant information based on factors like sentence importance and coherence. The paper also discusses the performance of different LLMs such as MPT-7b-instruct, falcon-7b-instruct, and OpenAI ChatGPT text-davinci-003 models in text summarization experiments conducted on datasets like CNN Daily Mail and XSum. Results show that text-davinci-003 outperformed others based on metrics like BLEU Score, Rouge Score, and Bert Score. Overall, this research serves as a valuable resource for researchers and practitioners in NLP by providing insights into the effectiveness of LLMs in text summarization across various datasets. It sets a foundation for developing advanced Generative AI applications that can tackle a wide range of business challenges effectively.