# MAIRA-1: A specialised large multimodal model for radiology report generation

**Authors:** Stephanie L. Hyland, Shruthi Bannur, Kenza Bouzid, Daniel C. Castro, Mercy Ranjit, Anton Schwaighofer, Fernando Pérez-García, Valentina Salvatelli, Shaury Srivastav, Anja Thieme, Noel Codella, Matthew P. Lungren, Maria Teodora Wetscherek, Ozan Oktay, Javier Alvarez-Valle  
**arXiv:** [2311.13668](https://arxiv.org/abs/2311.13668)  
**Date:** 2023-11-22  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2311.13668v1/

## Key points

- The authors present a radiology-specific multimodal model called MAIRA-1 for generating radiological reports from chest X-rays (CXRs)
- MAIRA-1 combines a CXR-specific image encoder with a fine-tuned large language model based on Vicuna-7B and text-based data augmentation techniques
- The proposed model achieves state-of-the-art quality in producing radiological reports
- It significantly improves on the radiologist-aligned RadCliQ metric as well as across all lexical metrics considered
- Manual review of the model outputs demonstrates promising fluency and accuracy of generated reports while uncovering failure modes that were not captured by existing evaluation practices
- More information and resources about MAIRA-1 can be found on their project website: https://aka.ms/maira

## Summary

The authors present a radiology-specific multimodal model called MAIRA-1 for generating radiological reports from chest X-rays (CXRs). The model leverages the idea that large language models can be equipped with multimodal capabilities by aligning them with pre-trained vision encoders. This alignment allows the model to gain image understanding and description capabilities, which is crucial for analyzing CXRs. MAIRA-1 combines a CXR-specific image encoder with a fine-tuned large language model based on Vicuna-7B and text-based data augmentation techniques. The proposed model achieves state-of-the-art quality in producing radiological reports. It significantly improves on the radiologist-aligned RadCliQ metric as well as across all lexical metrics considered. Manual review of the model outputs demonstrates promising fluency and accuracy of generated reports while uncovering failure modes that were not captured by existing evaluation practices. The authors provide more information and resources about MAIRA-1 on their project website: https://aka.ms/maira. The website likely contains additional details about the model architecture, training process, and evaluation results.