# Radiology-Llama2: Best-in-Class Large Language Model for Radiology

**Authors:** Zhengliang Liu, Yiwei Li, Peng Shu, Aoxiao Zhong, Longtao Yang, Chao Ju, Zihao Wu, Chong Ma, Jie Luo, Cheng Chen, Sekeun Kim, Jiang Hu, Haixing Dai, Lin Zhao, Dajiang Zhu, Jun Liu, Wei Liu, Dinggang Shen, Tianming Liu, Quanzheng Li  
**arXiv:** [2309.06419](https://arxiv.org/abs/2309.06419)  
**Date:** 2023-08-29  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2309.06419v1/

## Key points

- Radiology-Llama2 is a large language model specialized for radiology
- It is based on the Llama2 architecture and trained on a large dataset of radiology reports
- Radiology-Llama2 achieves state-of-the-art performance compared to other generative language models
- It outperforms all comparison models across all ROUGE metrics (ROUGE-1, ROUGE-2, and ROUGE-L) on both the MIMIC-CXR and OpenI datasets
- Radiology-Llama2 captures a higher proportion of overlapping unigrams and maintains content overlap in bigrams and longer sequences compared to Anthropic Claude2
- Baichuan 7B exhibits exceptionally low scores on both datasets, highlighting its limitations in capturing basic elements of content overlap
- Radiology-Llama2 understands the intention of text and delivers more comprehensive and clinically relevant impressions

## Summary

This paper introduces Radiology-Llama2, a large language model specialized for radiology through a process known as instruction tuning. Radiology-Llama2 is based on the Llama2 architecture and further trained on a large dataset of radiology reports to generate coherent and clinically useful impressions from radiological findings. The model achieves state-of-the-art performance compared to other generative language models, with a Rouge-1 score of 0.4834 on the MIMIC-CXR dataset and 0.4185 on the OpenI dataset, as evaluated using ROUGE metrics. Quantitative assessments demonstrate that Radiology-Llama2 significantly outperforms all comparison models across all ROUGE metrics (ROUGE-1, ROUGE-2, and ROUGE-L) on both datasets. For the MIMIC-CXR dataset, Radiology-Llama2 achieves higher scores than the second best performing model Anthropic Claude2 indicating its ability to capture a higher proportion of overlapping unigrams and maintain content overlap in bigrams and longer sequences. Similarly, on the OpenI dataset Radiology-Llama2 sustains its exemplary performance compared to Anthropic Claude2. The substantial gap between these two models across all metrics highlights Radiology-Llama2's robustness and generalizability across datasets. In contrast Baichuan 7B exhibits exceptionally low scores on both datasets emphasizing the limitations of such models in capturing even basic elements of content overlap. The paper provides specific examples where different LLMs are required to derive impressions from radiology findings in a report while some models understand the content but fail to capture important points or provide satisfactory answers; Radiology Llamas 2 stands out by understanding the intention of text and delivering more comprehensive and clinically relevant impressions.