# Neural Machine Translation of Rare Words with Subword Units

**Authors:** Rico Sennrich, Barry Haddow, Alexandra Birch  
**arXiv:** [1508.07909](https://arxiv.org/abs/1508.07909)  
**Date:** 2015-08-31  
**Summary page:** https://www.summarizepaper.com/en/paper/1508.07909v5/

## Key points

- Authors propose a new approach for translating rare and unknown words in NMT models
- Certain word classes can be translated more effectively through smaller units rather than whole words
- Different word segmentation techniques are discussed
- Subword models outperform a back-off dictionary baseline in English-German and English-Russian translation tasks
- Analysis of 100 rare tokens in German training data shows that the majority can potentially be translated using smaller units
- Segmenting rare words into appropriate subword units is sufficient for the NMT model to learn transparent translations and generalize this knowledge to translate unseen words
- Related work on handling unknown words in statistical machine translation is discussed
- Proposed approach offers a simpler and more effective solution for translating rare and unknown words in NMT models.

## Summary

In this paper, the authors propose a new approach for translating rare and unknown words in neural machine translation (NMT) models. They argue that certain word classes can be translated more effectively through smaller units rather than whole words and discuss different word segmentation techniques. Empirically demonstrating that subword models outperform a back-off dictionary baseline in English-German and English-Russian translation tasks, the authors analyze 100 rare tokens in their German training data and find that the majority of these tokens can potentially be translated from English using smaller units. The paper provides empirical support for the hypothesis that segmenting rare words into appropriate subword units is sufficient for the NMT model to learn transparent translations and generalize this knowledge to translate unseen words. Additionally, related work on handling unknown words in statistical machine translation is discussed. Overall, the proposed approach offers a simpler and more effective solution for translating rare and unknown words in NMT models.