# Back Translation for Speech-to-text Translation Without Transcripts

**Authors:** Qingkai Fang, Yang Feng  
**arXiv:** [2305.08709](https://arxiv.org/abs/2305.08709)  
**Date:** 2023-05-15  
**Summary page:** https://www.summarizepaper.com/en/paper/2305.08709v1/

## Key points

- Authors Qingkai Fang and Yang Feng address the challenge of enhancing end-to-end speech-to-text translation (ST) without relying on source transcripts.
- They propose a novel approach called back translation for speech-to-text translation (BT4ST) that leverages target-side monolingual data to improve ST performance.
- The BT4ST algorithm synthesizes pseudo ST data from monolingual target data, bypassing the need for source transcripts.
- To handle complexities in ST like short-to-long generation and one-to-many mapping, they incorporate self-supervised discrete units into their model.
- By cascading a target-to-unit model and a unit-to-speech model, they achieve back translation and generate synthetic ST data.
- Their study shows significant improvements in ST performance with an average boost of 2.3 BLEU on MuST-C En-De, En-Fr, and En-Es datasets.
- The method is effective in low-resource scenarios where transcripts are scarce or unavailable, showcasing its potential to bridge gaps in language processing tasks.
- Fang and Feng's innovative approach not only enhances ST capabilities but also opens up possibilities for improving language processing technologies in diverse linguistic contexts with limited or non-existent written transcripts.

## Summary

In their paper titled "Back Translation for Speech-to-text Translation Without Transcripts," authors Qingkai Fang and Yang Feng address the challenge of enhancing end-to-end speech-to-text translation (ST) without relying on source transcripts. They highlight the limitations posed by the lack of availability of transcripts, especially in unwritten languages globally. To overcome this hurdle, the authors propose a novel approach that leverages large amounts of target-side monolingual data to improve ST performance. Inspired by the success of back translation in machine translation (MT), Fang and Feng introduce a back translation algorithm specifically tailored for ST (BT4ST). This algorithm synthesizes pseudo ST data from monolingual target data, effectively bypassing the need for source transcripts. To tackle the complexities associated with short-to-long generation and one-to-many mapping in ST, they incorporate self-supervised discrete units into their model. By cascading a target-to-unit model and a unit-to-speech model, they achieve back translation and generate synthetic ST data. The results of their study demonstrate significant improvements in ST performance, with an average boost of 2.3 BLEU on MuST-C En-De, En-Fr, and En-Es datasets. Furthermore, additional experiments showcase the effectiveness of their method in low-resource scenarios, highlighting its potential to bridge gaps in language processing tasks where transcripts are scarce or unavailable. Overall, Fang and Feng's innovative approach not only enhances ST capabilities but also opens up possibilities for improving language processing technologies in diverse linguistic contexts where written transcripts may be limited or non-existent. Their research contributes valuable insights to the field of speech-to-text translation and sets a foundation for further advancements in overcoming transcription challenges in multilingual settings.