# Towards actionability for open medical imaging datasets: lessons from community-contributed platforms for data management and stewardship

**Authors:** Amelia Jiménez-Sánchez, Natalia-Rozalia Avlona, Dovile Juodelyte, Théo Sourget, Caroline Vang-Larsen, Hubert Dariusz Zając, Veronika Cheplygina  
**arXiv:** [2402.06353](https://arxiv.org/abs/2402.06353)  
**Date:** 2024-02-09  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.06353v1/

## Key points

- Medical imaging datasets are crucial for training and evaluating diagnostic algorithms in AI healthcare.
- Quality of these datasets impacts accuracy, robustness, and fairness of AI models.
- Concerns exist about governance models on Community-Contributed Platforms (CCPs) like Kaggle and HuggingFace regarding dataset sharing, documentation, and evaluation practices.
- Issues identified in popular datasets on CCPs include vague licenses, lack of persistent identifiers, duplicates, missing metadata, and platform discrepancies.
- Proposal for a commons-based stewardship model to enhance dataset documentation and maintenance on CCPs.
- Emphasis on tracking dataset evolution within static infrastructures to address biases or spurious correlations over time.
- Suggestions for creating living reviews through community-contributed platforms to enhance transparency and accountability in data development lifecycle.

## Summary

In the realm of artificial intelligence (AI) in healthcare, medical imaging datasets play a crucial role in training and evaluating diagnostic algorithms. The quality of these datasets directly impacts the accuracy, robustness, and fairness of AI models. While medical imaging datasets are increasingly accessible on Community-Contributed Platforms (CCPs) like Kaggle and HuggingFace, there are concerns regarding the governance model's ability to uphold recommended practices for sharing, documenting, and evaluating datasets. This paper delves into the documentation, sharing, and maintenance of medical imaging datasets on CCPs. Drawing distinctions between medical imaging and computer vision datasets, the study highlights potential downstream consequences of inadequate dataset management practices. An analysis of 20 popular datasets (10 medical and 10 computer vision) on CCPs reveals issues such as vague licenses, lack of persistent identifiers, duplicates, missing metadata, and platform discrepancies. Introducing the concept of "actionability" as a metric to assess data quality gaps on CCPs versus ideal characteristics for AI in healthcare applications, the paper proposes a commons-based stewardship model for enhancing dataset documentation and maintenance on these platforms. Additionally, it emphasizes the need for tracking dataset evolution within static infrastructures through processes that acknowledge data as dynamic entities that evolve over time. Recognizing challenges in capturing dataset changes without stable identifiers like DOIs or traditional reviews, the study underscores the importance of formalized practices for documenting explicit changes while addressing implicit variations that may introduce biases or spurious correlations. Suggestions include creating living reviews through community-contributed platforms where users can contribute derived datasets or related research to enhance transparency and accountability throughout the data development lifecycle. Acknowledging limitations based on quantitative evidence and subjective perceptions, the paper calls for more insights from all stakeholders involved in managing and stewarding medical imaging datasets to foster a comprehensive understanding of best practices in dataset documentation within AI-driven healthcare initiatives.