# No Language Left Behind: Scaling Human-Centered Machine Translation

**Authors:**  NLLB team, Marta R. Costa-jussà, James Cross, Onur Çelebi, Maha Elbayad, Kenneth Heafield, Kevin Heffernan, Elahe Kalbassi, Janice Lam, Daniel Licht, Jean Maillard, Anna Sun, Skyler Wang, Guillaume Wenzek, Al Youngblood, Bapi Akula, Loic Barrault, Gabriel Mejia Gonzalez, Prangthip Hansanti, John Hoffman  
**arXiv:** [2207.04672](https://arxiv.org/abs/2207.04672)  
**Date:** 2022-07-11  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2207.04672v1/

## Key points

- Researchers aim to address the challenge of language barriers on a global scale
- Machine translation has primarily focused on a small subset of languages, leaving behind low-resource languages
- Goal is to break the 200 language barrier while ensuring safe and high-quality translations with ethical considerations in mind
- Conducted exploratory interviews with native speakers to understand the need for low-resource language translation support
- Created datasets and models to narrow the performance gap between low and high-resource languages
- Developed a conditional compute model based on Sparsely Gated Mixture of Experts, trained on data obtained through novel data mining techniques tailored for low-resource languages
- Evaluated approach using Flores-200 benchmark and assessed translation safety using a novel toxicity benchmark covering all languages in Flores-200
- Achieved an improvement of 44% BLEU compared to previous state-of-the-art in over 40,000 different translation directions
- Proposed architectural and training improvements to counteract overfitting during training on thousands of tasks
- Discussed analysis of NLLB-200, out-of-domain generalization performance, making large models more accessible through distillation, including multiple scripts and related languoids as well as environmental impact considerations
- Research lays important groundwork towards realizing a universal translation system that can bridge language barriers worldwide

## Summary

In the study titled "No Language Left Behind: Scaling Human-Centered Machine Translation," the researchers aim to address the challenge of language barriers on a global scale. While machine translation has made significant progress, it has primarily focused on a small subset of languages, leaving behind low-resource languages. The goal is to break the 200 language barrier while ensuring safe and high-quality translations, with ethical considerations in mind. To achieve this, the researchers conducted exploratory interviews with native speakers to understand the need for low-resource language translation support. They then created datasets and models aimed at narrowing the performance gap between low and high-resource languages. Specifically, they developed a conditional compute model based on Sparsely Gated Mixture of Experts, trained on data obtained through novel data mining techniques tailored for low-resource languages. To evaluate their approach, the researchers used a human-translated benchmark called Flores-200 and assessed translation safety using a novel toxicity benchmark covering all languages in Flores-200. They evaluated over 40,000 different translation directions and achieved an improvement of 44% BLEU (Bilingual Evaluation Understudy) compared to the previous state-of-the-art. The study also proposes multiple architectural and training improvements to counteract overfitting during training on thousands of tasks. Additionally, they discuss analysis of NLLB-200 (No Language Left Behind - 200), out-of domain generalization performance making large models more accessible through distillation effectively including languages with multiple scripts and related languoids as well as environmental impact considerations. Overall, this research lays important groundwork towards realizing a universal translation system that can bridge language barriers worldwide. The contributions described in this work are open sourced and accessible for further exploration.