# Phi-3 Technical Report: A Highly Capable Language Model Locally on Your Phone

**Authors:** Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla, Nguyen Bach, Amit Bahree, Arash Bakhtiari, Harkirat Behl, Alon Benhaim, Misha Bilenko, Johan Bjorck, Sébastien Bubeck, Martin Cai, Caio César Teodoro Mendes, Weizhu Chen, Vishrav Chaudhary, Parul Chopra, Allie Del Giorno  
**arXiv:** [2404.14219](https://arxiv.org/abs/2404.14219)  
**Date:** 2024-04-22  
**Summary page:** https://www.summarizepaper.com/en/paper/2404.14219v1/

## Key points

- Phi-3-mini is a 3.8 billion parameter language model trained on a dataset of 3.3 trillion tokens
- Phi-3-mini competes with leading models like Mixtral 8x7B and GPT-3.5, achieving scores of 69% on MMLU and 8.38 on MT-bench
- Key innovation of phi-3-mini is its training dataset, which enhances robustness, safety features, and chat format capabilities
- Introduction of larger models phi-3-small and phi-3-medium trained on 4.8 trillion tokens each with enhanced capabilities, scoring 75% and 78% on MMLU respectively
- Team includes talented individuals like Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan among others pushing boundaries in language modeling technology

## Summary

The Phi-3 Technical Report introduces the phi-3-mini, a groundbreaking 3.8 billion parameter language model trained on an extensive dataset of 3.3 trillion tokens. This model's exceptional performance has been validated by both academic benchmarks and internal evaluations, placing it in direct competition with leading models such as Mixtral 8x7B and GPT-3.5. Impressively, the phi-3-mini achieves remarkable scores of 69% on MMLU and 8.38 on MT-bench while maintaining a compact size suitable for deployment on mobile devices. A key innovation of the phi-3-mini lies in its training dataset, which is an enhanced version of the one utilized for its predecessor, the phi-2 model. This meticulously filtered web data and synthetic information contribute to the model's robustness, safety features, and optimized chat format capabilities. Furthermore, the report delves into additional advancements with the introduction of two larger models: phi-3-small and phi-3-medium. These models are trained on a staggering 4.8 trillion tokens each and exhibit significantly enhanced capabilities compared to the phi-3-mini. For instance, they achieve impressive scores of 75% and 78% on MMLU respectively while scoring 8.7 and 8.9 on MT-bench. The team behind this groundbreaking research includes a diverse group of talented individuals such as Marah Abdin, Sam Ade Jacobs, Ammar Ahmad Awan, Jyoti Aneja, Ahmed Awadallah, Hany Awadalla among others who have collectively contributed to pushing the boundaries of language modeling technology. In conclusion, showcases a new era in language modeling technology with its innovative approach to training datasets and cutting-edge models that promise to revolutionize natural language processing capabilities across various applications.