# PaLM 2 Technical Report

**Authors:** Rohan Anil, Andrew M. Dai, Orhan Firat, Melvin Johnson, Dmitry Lepikhin, Alexandre Passos, Siamak Shakeri, Emanuel Taropa, Paige Bailey, Zhifeng Chen, Eric Chu, Jonathan H. Clark, Laurent El Shafey, Yanping Huang, Kathy Meier-Hellstern, Gaurav Mishra, Erica Moreira, Mark Omernick, Kevin Robinson, Sebastian Ruder  
**arXiv:** [2305.10403](https://arxiv.org/abs/2305.10403)  
**Date:** 2023-05-17  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.10403v1/

## Key points

- PaLM 2 is a new state-of-the-art language model that surpasses its predecessor, PaLM, in multilingual and reasoning capabilities while being more compute-efficient.
- It is a Transformer-based model trained using a mixture of objectives.
- PaLM 2 exhibits significantly improved quality on downstream tasks across different model sizes.
- It offers faster and more efficient inference compared to PaLM, enabling broader deployment and allowing for a more natural pace of interaction.
- PaLM 2 showcases robust reasoning capabilities with substantial improvements over PaLM on BIG-Bench and other reasoning tasks.
- It maintains stable performance on responsible AI evaluations and provides inference-time control over toxicity without additional overhead or impact on other capabilities.
- When discussing the PaLM 2 family, it is important to differentiate between pre-trained models of different sizes, fine-tuned variants, and user-facing products that utilize them.
- The report delves into responsible usage of PaLM 2, particularly focusing on inference time control, with recommendations for developers to ensure responsible development practices.
- Overall, PaLM 2 achieves state-of-the-art performance across various tasks and capabilities.

## Summary

We introduce PaLM 2, a new state-of-the-art language model that surpasses its predecessor, PaLM, in terms of multilingual and reasoning capabilities while being more compute-efficient. PaLM 2 is a Transformer-based model trained using a mixture of objectives. Extensive evaluations on English and multilingual language and reasoning tasks demonstrate that PaLM 2 exhibits significantly improved quality on downstream tasks across different model sizes. Moreover, it offers faster and more efficient inference compared to PaLM, enabling broader deployment and allowing for a more natural pace of interaction. PaLM 2 showcases robust reasoning capabilities with substantial improvements over PaLM on BIG-Bench and other reasoning tasks. It also maintains stable performance on responsible AI evaluations and provides inference-time control over toxicity without additional overhead or impact on other capabilities. Overall, PaLM 2 achieves state-of-the-art performance across various tasks and capabilities. When discussing the PaLM 2 family, it is important to differentiate between pre-trained models of different sizes, fine-tuned variants of these models, and the user-facing products that utilize them. User-facing products often involve additional pre- and postprocessing steps while the underlying models may evolve over time; therefore exact performance matching between user facing products and results reported in this report should not be expected. The report also delves into responsible usage of PaLM 2 particularly focusing on inference time control; recommendations for developers are provided to ensure responsible development practices. In conclusion, PaLM 2 represents a significant advancement in language modeling with enhanced multilingualism, reasoning abilities efficiency as well as responsible AI features. The detailed results section further explores scaling laws experiments training datasets used for evaluation purposes as well as evaluations conducted for language proficiency exams classification/question answering tasks reasoning tasks (such as coding) translation tasks (including natural language generation) memorization analysis among others. The report concludes with authorship details along with acknowledgements providing additional sections on responsible AI including dataset analysis evaluation approaches dialog uses classification uses translation uses question answering uses language modeling measurement quality rubrics CrowdWorksheets usage and a model card.