# Direct Speech Translation for Automatic Subtitling

**Authors:** Sara Papi, Marco Gaido, Alina Karakanta, Mauro Cettolo, Matteo Negri, Marco Turchi  
**arXiv:** [2209.13192](https://arxiv.org/abs/2209.13192)  
**Date:** 2022-09-27  
**Summary page:** https://www.summarizepaper.com/en/paper/2209.13192v1/

## Key points

- Automatic subtitling involves translating speech into timed text while adhering to multiple space and time requirements
- Traditionally, this task has been addressed through a pipeline of separate elements dealing with transcription, translation, segmentation into subtitles, and predicting timestamps
- The authors propose the first direct automatic subtitling model that generates target language subtitles and their timestamps from the source speech in a single solution
- The system achieved better or competitive results compared to state-of-the-art cascaded models for named entities (NE) and numbers using EuroParl Interviews for testing
- The system struggled more with PERSON recognition compared to cascaded systems
- The authors tested their system on EC Short Documentaries test set and found that there was no clear winner among the systems since each model performed differently for different languages
- Overall, the authors' proposed system represents a significant step towards developing high-performing direct models for automatic subtitling without employing any ad-hoc decoding or training strategy aimed at reducing translation length or compressing information.

## Summary

Automatic subtitling is a complex task that involves translating speech from an audiovisual product into short pieces of timed text, or subtitles, while adhering to multiple space and time requirements. Traditionally, this task has been addressed through a pipeline of separate elements dealing with transcription, translation, segmentation into subtitles, and predicting timestamps. However, in a recent paper, the authors propose the first direct automatic subtitling model that generates target language subtitles and their timestamps from the source speech in a single solution. To evaluate the performance of their system compared to state-of-the-art cascaded models trained with both in- and out-domain data, the authors conducted an output analysis on named entities (NE) and numbers using EuroParl Interviews for testing. The results showed that their system achieved better or competitive results compared to System 4 for GPE and ORG in en-de and for ORG in en-es. However, it struggled more with PERSON recognition compared to cascaded systems. The authors also tested their system on EC Short Documentaries test set and found that there was no clear winner among the systems since each model performed differently for different languages. Their direct system performed well considering production tools: reaching second-best quality for BLEU in German and third-best in Spanish; third-best score for Sigma in German and fifth-best in Spanish; performing better than System 2 and 5 for SubER in both languages. Overall, the authors' proposed system represents a significant step towards developing high-performing direct models for automatic subtitling without employing any ad-hoc decoding or training strategy aimed at reducing translation length or compressing information. While there is still room for improvement regarding accuracy concerning named entities' recognition given its role in human understanding's importance, this study shows promising results.