# Emergent Abilities of Large Language Models

**Authors:** Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, William Fedus  
**arXiv:** [2206.07682](https://arxiv.org/abs/2206.07682)  
**Date:** 2022-06-15  
**Summary page:** https://www.summarizepaper.com/en/paper/2206.07682v1/

## Key points

- Scaling up language models improves performance and sample efficiency
- Emergent abilities only appear in larger models and cannot be predicted by smaller models
- Further scaling could expand the capabilities of language models
- Emergent abilities can span different types of language models, tasks, and experimental scenarios
- More undiscovered emergent abilities may exist as language models continue to scale
- Specific examples of emergent abilities include finetuning to predict intermediate outputs for executing multi-step computations
- The study emphasizes the importance of further research into understanding how these abilities emerge and the potential for additional scaling in natural language processing.

## Summary

This paper explores the concept of emergent abilities in large language models. While scaling up language models has been shown to improve performance and sample efficiency on various tasks, emergent abilities are unpredictable phenomena that only appear in larger models. These abilities cannot be predicted by extrapolating the performance of smaller models. The existence of emergent abilities suggests that further scaling could expand the capabilities of language models even more. The authors discuss how emergent abilities can span different types of language models, tasks, and experimental scenarios. They highlight that these abilities have only been observed at a certain computational scale, indicating that there may be more undiscovered emergent abilities as language models continue to scale. The paper also touches upon specific examples of emergent abilities such as finetuning language models to predict intermediate outputs which enables them to successfully execute multi-step computations like adding large numbers or executing computer programs. In conclusion, this study sheds light on the emergence of new capabilities in large language models and emphasizes the importance of further research into understanding how these abilities emerge and whether additional scaling can unlock even more potential in natural language processing.