# Contextual Refinement of Translations: Large Language Models for Sentence and Document-Level Post-Editing

**Authors:** Sai Koneru, Miriam Exel, Matthias Huck, Jan Niehues  
**arXiv:** [2310.14855](https://arxiv.org/abs/2310.14855)  
**Date:** 2023-10-23  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.14855v1/

## Key points

- Investigating use of Large Language Models (LLM's) for Neural Machine Translation (NMT)
- Initial experiments showed performance degradation when fine-tuning LLM's for translation
- Proposing adapting LLM's as Automatic Post-Editors (APE) instead of direct translators
- Introducing Low-Rank-Adapter fine-tuning for APEs, leading to significant improvements in metrics and out-of-domain data generalization
- Achieving state-of-the-art accuracy rate of 89% on ContraPro test set assessing pronoun ambiguities in English to German translation
- Demonstrating the effectiveness of manual post-editing for document-level translation with reference context provided
- Exploring Chunk-Based and Batched Sliding Window approaches to enhance translation process
- Highlighting potential of using LLM's as APEs to improve translation quality at both sentence and document levels

## Summary

In this study, we investigate the use of Large Language Models (LLM's) for Neural Machine Translation (NMT) and explore parameter-efficient fine-tuning techniques. While LLM's have shown success in various Natural Language Processing tasks, they have not yet achieved state-of-the-art performance in NMT. Our initial experiments found that fine-tuning for translation purposes led to performance degradation. To address this issue, we propose adapting LLM's as Automatic Post-Editors (APE) rather than direct translators. By leveraging their exceptional ability to process and generate lengthy sequences, we extend our approach to document-level translation. We also introduce Low-Rank-Adapter fine-tuning for APEs and observe significant improvements across both sentence and document-level metrics, even generalizing to out-of-domain data. Notably, our method achieves a state-of-the-art accuracy rate of 89% on the ContraPro test set which specifically assesses pronoun ambiguities when translating from English to German. Furthermore, we investigate a practical scenario involving manual post-editing for document-level translation with reference context provided. Our results demonstrate that incorporating human corrections can significantly reduce the number of edits required for subsequent translations. We explore various approaches such as Chunk-Based and Batched Sliding Window to enhance the translation process. Overall, our study highlights the potential of using LLM's as APEs for improving translation quality at both sentence and document levels. The findings suggest that integrating manual feedback can further enhance the efficiency and accuracy of machine translation systems.