# FLEUR: An Explainable Reference-Free Evaluation Metric for Image Captioning Using a Large Multimodal Model

**Authors:** Yebin Lee, Imseong Park, Myungjoo Kang  
**arXiv:** [2406.06004](https://arxiv.org/abs/2406.06004)  
**Date:** 2024-06-10  
**Summary page:** https://www.summarizepaper.com/en/paper/2406.06004v1/

## Key points

- Authors address limitations of existing image captioning evaluation metrics
- Focus on assigning a single numerical score by comparing with reference captions
- Lack explainability in assigned score and require expensive reference captions
- Proposal of FLEUR as a novel reference-free metric for image captioning evaluation
- Introduces explainability into evaluation metrics
- Leverages large multimodal model to evaluate captions against images without reference captions
- Provides explanations for assigned scores
- Introduction of score smoothing techniques to align with human judgment and ensure robustness to user-defined grading criteria
- FLEUR demonstrates high correlations with human judgment across various image captioning benchmarks
- Achieves state-of-the-art results on datasets such as Flickr8k-CF, COMPOSITE, and Pascal-50S within the domain of reference-free evaluation metrics
- Source code and results publicly available at https://github.com/Yebin46/FLEUR

## Summary

In their paper titled "FLEUR: An Explainable Reference-Free Evaluation Metric for Image Captioning Using a Large Multimodal Model," authors Yebin Lee, Imseong Park, and Myungjoo Kang address the limitations of existing image captioning evaluation metrics that focus on assigning a single numerical score to a caption by comparing it with reference captions. These methods lack explainability in the assigned score and require expensive reference captions. To overcome these challenges, the authors propose FLEUR, a novel reference-free metric that introduces explainability into image captioning evaluation metrics. By leveraging a large multimodal model, FLEUR can evaluate captions against images without the need for reference captions and provide explanations for the assigned scores. The authors also introduce score smoothing techniques to align closely with human judgment and ensure robustness to user-defined grading criteria. Through extensive experimentation, FLEUR demonstrates high correlations with human judgment across various image captioning evaluation benchmarks. Furthermore, FLEUR achieves state-of-the-art results on popular datasets such as Flickr8k-CF, COMPOSITE, and Pascal-50S within the domain of reference-free evaluation metrics. The authors have made their source code and results publicly available at https://github.com/Yebin46/FLEUR. This research was accepted at ACL (Main) 2024 and contributes significantly to advancing explainable and accurate evaluation metrics in the field of image captioning.