# Do We Need Another Explainable AI Method? Toward Unifying Post-hoc XAI Evaluation Methods into an Interactive and Multi-dimensional Benchmark

**Authors:** Mohamed Karim Belaid, Eyke Hüllermeier, Maximilian Rabus, Ralf Krestel  
**arXiv:** [2207.14160](https://arxiv.org/abs/2207.14160)  
**Date:** 2022-06-08  
**Summary page:** https://www.summarizepaper.com/en/paper/2207.14160v2/

## Key points

- Explanation as a legal right has gained attention in recent years
- Importance of going beyond accuracy metrics to improve AI models by debugging learned patterns and demystifying AI behavior
- Challenges arising from the widespread use of explainable AI (XAI)
- Increase in published XAI algorithms, making it difficult for practitioners to choose the most suitable tool
- Potential misuse and misinterpretation of XAI algorithms by data scientists
- Proposal of a benchmark led by Mohamed Karim Belaid, Eyke Hüllermeier, Maximilian Rabus, and Ralf Krestel to address issues and ensure proper comparison and utilization of XAI
- Benchmark aims to unify exclusive functional testing methods for XAI algorithms
- Development of a selection protocol to identify non-redundant functional tests tailored to meet specific end-user requirements for explaining a model
- Hierarchical scoring system with three levels targeting different end-user groups: researchers, practitioners, and laymen in XAI
- Grouping tests into five categories: fidelity, fragility, stability, simplicity, and stress tests
- Inclusion of an aggregated comprehensibility score simplifying interpreting algorithm outputs into one easy-to-compare value
- Interactive user interface aiming to recommend suitable solutions for various machine learning tasks along with their current limitations

## Summary

In recent years, the concept of  has gained significant attention as countries have recognized explanations as a legal right. This goes beyond just accuracy metrics and allows for the improvement of models by debugging learned patterns and demystifying AI behavior. However, with the widespread use of , new challenges have emerged. On one hand, there has been an increase in published  algorithms, making it difficult for practitioners to choose the most suitable tool. On the other hand, some experiments have shown how easily data scientists can misuse these algorithms and misinterpret their results. To address these issues and ensure proper comparison and utilization of , a team led by Mohamed Karim Belaid, Eyke Hüllermeier, Maximilian Rabus, and Ralf Krestel has proposed . This benchmark aims to unify all exclusive functional testing methods applied to  algorithms. The team has developed a selection protocol to identify non-redundant functional tests from existing literature that are tailored to meet specific end-user requirements for explaining a model. encapsulates the complexity of evaluating  methods through a hierarchical scoring system with three levels targeting different end-user groups: researchers, practitioners, and laymen in . At the most detailed level, individual scores are provided for each test while grouping them into five categories - fidelity, fragility, stability, simplicity, and stress tests - at the second level. The benchmark also includes an aggregated comprehensibility score that simplifies interpreting algorithm outputs into one easy-to-compare value. The interactive user interface of  aims to mitigate errors in interpreting  results by quickly recommending suitable solutions for various machine learning tasks along with their current limitations. This comprehensive benchmark is now accessible at https://karim-53.github.io/cxai/, offering a valuable resource for those navigating the complex landscape of explainable AI methods.