# A Paradigm Shift in Machine Translation: Boosting Translation Performance of Large Language Models

**Authors:** Haoran Xu, Young Jin Kim, Amr Sharaf, Hany Hassan Awadalla  
**arXiv:** [2309.11674](https://arxiv.org/abs/2309.11674)  
**Date:** 2023-09-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2309.11674v1/

## Key points

- Significant advancements in various tasks have not fully translated to the translation task for models with moderate sizes
- Previous attempts to improve translation capabilities of moderate-sized language models (LLMs) have had limited success
- The authors propose a novel approach specifically designed for the translation task, eliminating the need for abundant parallel data
- The proposed approach consists of two stages of fine-tuning: initial fine-tuning on monolingual data and subsequent fine-tuning on a small set of high-quality parallel data
- Experimental results show that ALMA, a developed LLM based on LLaMA-2, achieves significant improvements in translation performance across ten directions from test datasets
- ALMA outperforms previous work and even surpasses models with larger parameters like NLLB-54B and GPT-3.5-text-davinci-003
- This study demonstrates that fine-tuning approaches can enhance translation capabilities of moderate-sized LLMs without relying heavily on parallel data
- The proposed method opens up new possibilities for improving machine translation performance using large language models.

## Summary

In recent years,  have made significant advancements in various  tasks. However, these advancements have not been fully reflected in the translation task, particularly for models with moderate sizes (i.e., 7B or 13B parameters). These moderate LLMs still lag behind conventional supervised encoder-decoder translation models. Previous studies have attempted to improve the translation capabilities of these moderate LLMs but with limited success. To address this issue, the authors propose a novel  specifically designed for the translation task. This approach eliminates the need for abundant parallel data that traditional translation models typically rely on. The proposed approach consists of two stages of fine-tuning: initial fine-tuning on monolingual data and subsequent fine-tuning on a small set of high-quality parallel data. The authors introduce their developed LLM, called , which is based on LLaMA-2 as the underlying model. Experimental results demonstrate that ALMA achieves an average improvement of more than 12 BLEU and 12 COMET over its zero-shot performance across ten translation directions from the WMT'21 and WMT'22 test datasets. Notably, ALMA's performance surpasses all previous work and even outperforms models like NLLB-54B and GPT-3.5-text-davinci-003, despite having only 7B or 13B parameters. This study establishes a foundation for a novel training paradigm in machine translation by demonstrating that fine-tuning approaches can significantly enhance the translation capabilities of moderate-sized LLMs without relying heavily on parallel data. The proposed method opens up new possibilities for improving machine translation performance using large language models.