# XAI-TRIS: Non-linear image benchmarks to quantify false positive post-hoc attribution of feature importance

**Authors:** Benedict Clark, Rick Wilming, Stefan Haufe  
**arXiv:** [2306.12816](https://arxiv.org/abs/2306.12816)  
**Date:** 2023-06-22  
**Summary page:** https://www.summarizepaper.com/en/paper/2306.12816v2/

## Key points

- Focus on developing methods to make complex machine learning models more understandable to humans
- Attribution of 'importance' scores to input features for interpreting model predictions
- Lack of formal underpinning and challenges in drawing reliable conclusions from results
- Crafting explanations for different classification scenarios with known important features as ground truth
- Evaluation of explanation performance using quantitative metrics across deep learning model architectures
- Popular explanation methods struggle to outperform random baselines and may attribute false importance to irrelevant features
- Variability in explanations derived from different model architectures, leading to potential misinterpretations
- Emphasis on rigorous evaluation and standardization in explainable AI for reliable and consistent explanations
- Identification of suppressor variables that can influence model predictions by removing noise signals

## Summary

In the field of , there has been a focus on developing methods that aim to make the decisions of complex machine learning (ML) models more understandable to humans. One common approach is to attribute 'importance' scores to input features, which can help users interpret how a model arrived at its predictions. However, despite the popularity of , there is a lack of formal underpinning that makes it challenging to draw reliable conclusions from their results. This gap also hinders the theoretical verification and empirical validation of , particularly when dealing with challenging non-linear problems typically addressed by deep neural networks. To address these issues, researchers have crafted  for different classification scenarios where the important class-conditional features are known by design, serving as ground truth explanations. By using novel quantitative metrics, they have evaluated the explanation performance of various  across different deep learning model architectures. Surprisingly, the study revealed that popular  often struggle to outperform random performance baselines and edge detection techniques. In some cases, these methods mistakenly attribute false-positive importance to features that have no statistical relationship with the prediction target instead of focusing on truly important features. Moreover, the research highlights that explanations derived from different model architectures can vary significantly, leading to potential misinterpretations even under controlled conditions. This emphasizes the need for more rigorous evaluation and standardization in  to ensure reliable and consistent explanations across diverse applications. The study also sheds light on suppressor variables – features that do not directly correlate with the prediction target but can still influence model predictions by removing unwanted noise signals. Overall, this work underscores the importance of critically assessing  and their ability to provide meaningful insights into complex ML models. By establishing clear criteria for what constitutes an important feature and conducting thorough evaluations using , researchers can enhance the transparency and reliability of explainable AI systems in tackling non-linear problems effectively.