# Google USM: Scaling Automatic Speech Recognition Beyond 100 Languages

**Authors:** Yu Zhang, Wei Han, James Qin, Yongqiang Wang, Ankur Bapna, Zhehuai Chen, Nanxin Chen, Bo Li, Vera Axelrod, Gary Wang, Zhong Meng, Ke Hu, Andrew Rosenberg, Rohit Prabhavalkar, Daniel S. Park, Parisa Haghani, Jason Riesa, Ginger Perng, Hagen Soltau, Trevor Strohman  
**arXiv:** [2303.01037](https://arxiv.org/abs/2303.01037)  
**Date:** 2023-03-02  
**Summary page:** https://www.summarizepaper.com/en/paper/2303.01037v3/

## Key points

- The Universal Speech Model (USM) is a large-scale model developed by Google for automatic speech recognition (ASR) in over 100 languages.
- The USM's encoder is pre-trained on a massive unlabeled multilingual dataset of 12 million hours of audio data from 300 languages.
- Fine-tuning on a smaller labeled dataset helps the USM achieve state-of-the-art performance in multilingual ASR and speech-to-text translation tasks.
- Techniques like multilingual pre-training with random projection quantization and speech text modality matching are used to improve results.
- Despite using a smaller labeled training set compared to the Whisper model, the USM performs comparably or better in both in-domain and out-of-domain speech recognition tasks across multiple languages.
- The paper about the USM is 20 pages long and includes 7 figures and 8 tables.

## Summary

The Universal Speech Model (USM) is a large-scale model developed by Google that enables automatic speech recognition (ASR) across more than 100 languages. The model's encoder is pre-trained on a massive unlabeled multilingual dataset consisting of 12 million hours of audio data from over 300 languages. This pre-training is followed by fine-tuning on a smaller labeled dataset to achieve state-of-the-art performance on multilingual ASR and speech-to-text translation tasks. To further improve the results, the USM utilizes several techniques such as multilingual pre-training with random projection quantization and speech text modality matching. Despite using a labeled training set that is only 1/7th the size of the Whisper model's training set, the USM demonstrates comparable or even better performance in both in domain and out of domain speech recognition tasks across numerous languages. The paper consists of 20 pages and includes 7 figures and 8 tables.