# Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation

**Authors:** Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, Young Jin Kim  
**arXiv:** [2401.08417](https://arxiv.org/abs/2401.08417)  
**Date:** 2024-01-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2401.08417v1/

## Key points

- Study explores the performance of moderate-sized large language models (LLMs) in machine translation (MT)
- Aims to bridge the performance gap between LLMs with 7B or 13B parameters and state-of-the-art conventional encoder-decoder models or larger-scale LLMs
- Proposes a novel approach called Contrastive Preference Optimization (CPO) to address shortcomings of supervised fine-tuning (SFT) for LLMs in MT task
- CPO trains models to generate high-quality translations instead of mimicking reference translations like SFT does
- Applied CPO to ALMA models with 22K parallel sentences and 12M parameters, resulting in significant improvements compared to traditional supervised fine-tuning
- ALMA-R model demonstrates performance on par with or surpassing WMT competition winners and GPT-4 on WMT'21, WMT'22, and WMT'23 test datasets
- Introduces a new approach to optimize LLMs for machine translation tasks by training them to generate high-quality translations
- Findings showcase how Contrastive Preference Optimization can push the boundaries of LLM performance in machine translation.

## Summary

In this study titled "Contrastive Preference Optimization: Pushing the Boundaries of LLM Performance in Machine Translation," Haoran Xu, Amr Sharaf, Yunmo Chen, Weiting Tan, Lingfeng Shen, Benjamin Van Durme, Kenton Murray, and Young Jin Kim explore the performance of moderate-sized large language models (LLMs) in machine translation (MT). The researchers aim to bridge the performance gap between LLMs with 7B or 13B parameters and state-of-the-art conventional encoder-decoder models or larger-scale LLMs such as GPT-4. They address the shortcomings of supervised fine-tuning (SFT) for LLMs in the MT task by proposing a novel approach called Contrastive Preference Optimization (CPO). CPO trains models to generate high-quality translations instead of mimicking reference translations like SFT does. To evaluate its effectiveness, CPO is applied to ALMA models with only 22K parallel sentences and 12M parameters. The results show significant improvements compared to traditional supervised fine-tuning. The resulting model, called ALMA-R, demonstrates performance on par with or surpassing that of WMT competition winners and GPT-4 on WMT'21, WMT'22, and WMT'23 test datasets. This study introduces a new approach to optimize LLMs for machine translation tasks by training them to generate high-quality translations rather than just adequate ones. The findings showcase how Contrastive Preference Optimization can push the boundaries of LLM performance in machine translation.