# Explainable Artificial Intelligence for Human Decision-Support System in Medical Domain

**Authors:** Samanta Knapič, Avleen Malhi, Rohit Saluja, Kary Främling  
**arXiv:** [2105.02357](https://arxiv.org/abs/2105.02357)  
**Date:** 2021-05-05  
**Summary page:** https://www.summarizepaper.com/en/paper/2105.02357v1/

## Key points

- Exploring the potential of Explainable Artificial Intelligence (XAI) methods for decision-support in medical image analysis scenarios
- Focusing on improving the comprehensibility of decisions made by a Convolutional Neural Network (CNN) through visual explanations
- Applying three types of explainable methods - LIME, SHAP, and CIU - to a dataset of in-vivo gastral images obtained from Video capsule endoscopy (VCE)
- Goal: Increase trust among health professionals in CNN predictions
- Conducted three user studies using human evaluation with participants from non-medical backgrounds
- Findings suggest that the CIU method outperformed LIME and SHAP in terms of increasing support for human decision-making, transparency, and understandability
- CIU generated explanations more rapidly than the other two methods
- Future evaluations should involve domain experts and focus on application-grounded assessments specific to the use of explanations
- Expanding scope by testing explanations on more complex medical cases and applying them in real-life settings would enhance applicability
- Increasing sample size would provide more statistically meaningful outcomes
- Plan to generalize these explainable methods on different medical datasets with future improvements in implementation for decision-support in various scenarios

## Summary

In this paper, we explore the potential of Explainable Artificial Intelligence (XAI) methods for decision-support in medical image analysis scenarios. Specifically, we focus on improving the comprehensibility of decisions made by a Convolutional Neural Network (CNN) by providing visual explanations. We apply three types of explainable methods - LIME, SHAP, and CIU - to a dataset of in-vivo gastral images obtained from Video capsule endoscopy (VCE). The goal is to increase trust among health professionals in the predictions made by the CNN. To evaluate the effectiveness of these explainable methods, we conducted three user studies using human evaluation. Participants from non-medical backgrounds were asked to provide feedback on their experience and understanding of the explanations provided by LIME, SHAP, and CIU. Each user group consisted of 20 participants. Our findings suggest that the CIU method outperformed both LIME and SHAP in terms of increasing support for human decision-making and being more transparent and understandable to users. Additionally, CIU generated explanations more rapidly than the other two methods. These results indicate that there are notable differences in human decision-making depending on the type of explanation support provided. Future evaluations should involve domain experts and focus on application-grounded assessments specific to the use of explanations. Moreover, expanding scope by testing these explanations on more complex medical cases as well as applying them in real-life settings would enhance their applicability. Increasing sample size would also provide more statistically meaningful outcomes. Moving forward, we plan to generalize these explainable methods on different medical datasets with future improvements in implementation; this would provide decision-support for medical experts in a wide range of scenarios. In conclusion, our research offers valuable insights into the details of explanation support and its potential implementation in machine learning. The CIU method shows promise as a highly effective explainable method, providing decision-support and improving performance.