# An empirical study of the effect of background data size on the stability of SHapley Additive exPlanations (SHAP) for deep learning models

**Authors:** Han Yuan, Mingxuan Liu, Lican Kang, Chenkui Miao, Ying Wu  
**arXiv:** [2204.11351](https://arxiv.org/abs/2204.11351)  
**Date:** 2022-04-24  
**Summary page:** https://www.summarizepaper.com/en/paper/2204.11351v3/

## Key points


## Summary

The interpretation of machine learning (ML) models is becoming increasingly important, and SHapley Additive exPlanations (SHAP) is one external method that provides both instance and model-level explanations for deep learning (DL) models. However, the effect of background dataset size on SHAP's stability remains unexplored. In an empirical study using the MIMIC-III dataset, researchers found that SHAP values and variable rankings fluctuate when using different background datasets acquired from random sampling, indicating that users cannot blindly trust SHAP's interpretation. The fluctuations decrease with an increase in the background dataset size. Additionally, there is a U-shape in the stability assessment of SHAP variable rankings, demonstrating that it is more reliable in ranking the most and least important variables compared to moderately important ones. Overall, this study suggests that users should consider how background data affects SHAP results and opt for larger dataset sizes to mitigate fluctuations in SHAP's stability. To ensure reliable results from their interpretations of ML models using SHAP, users should use larger datasets when possible. The code used in this study is publicly accessible.