# Fine-tuning Large Language Models for Domain-specific Machine Translation

**Authors:** Jiawei Zheng, Hanghai Hong, Feiyan Liu, Xiaoli Wang, Jingsong Su, Yonggui Liang, Shikai Wu  
**arXiv:** [2402.15061](https://arxiv.org/abs/2402.15061)  
**Date:** 2024-02-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.15061v2/

## Key points

- Authors explore the potential of large language models (LLMs) in domain-specific machine translation (MT)
- Significant challenge faced by LLMs pre-trained on general domain corpora - lack of domain-specific knowledge leading to poor generalization
- Introduction of DragFT, a novel fine-tuning framework for enhancing domain-specific MT capabilities of LLMs
- Dictionary-enhanced prompting integrates dictionary information into prompts
- RAG-based few-shot example selection provides high-quality examples mimicking domain and style characteristics
- Fine-tuning with few-shot examples further boosts performance using in-domain examples
- Validation of DragFT's effectiveness on three well-known LLM backbones with 13B training parameters, outperforming advanced models like GPT-3.5 and GPT-4o
- Success attributed to DragFT's ability to incorporate relevant knowledge while minimizing noise in the fine-tuning process

## Summary

In their paper titled "Fine-tuning Large Language Models for Domain-specific Machine Translation," authors Jiawei Zheng, Hanghai Hong, Feiyan Liu, Xiaoli Wang, Jingsong Su, Yonggui Liang, and Shikai Wu explore the potential of large language models (LLMs) in domain-specific machine translation (MT). They highlight a significant challenge faced by LLMs pre-trained on general domain corpora - the lack of domain-specific knowledge leading to poor generalization to specific domains. To tackle this issue, the authors introduce DragFT, a novel fine-tuning framework aimed at enhancing the domain-specific MT capabilities of LLMs. DragFT employs three key techniques to augment LLMs: 
1. Dictionary-enhanced prompting integrates dictionary information into prompts to enhance the translation of domain-specific terminology. 2. RAG-based few-shot example selection provides high-quality examples that mimic both domain and style characteristics. 3. Fine-tuning with few-shot examples further boosts performance when utilizing in-domain examples. The authors validate DragFT's effectiveness by deploying it on three well-known LLM backbones with 13B training parameters. Their experiments on three domain-specific datasets demonstrate that DragFT significantly improves performance and outperforms advanced models like GPT-3.5 and GPT-4o. < kd>Few-Shot Examples</ kd>< kd>In-Domain Examples</ kd>< kd>GPT Models Comparison</ kd>

The success of DragFT is attributed to its ability to incorporate relevant knowledge while minimizing noise in the fine-tuning process. Overall, this study sheds light on the importance of enhancing LLMs for domain-specific MT tasks and presents a promising solution in DragFT that can lead to substantial performance improvements in translating specialized content accurately and effectively.