# Interpretability Needs a New Paradigm

**Authors:** Andreas Madsen, Himabindu Lakkaraju, Siva Reddy, Sarath Chandar  
**arXiv:** [2405.05386](https://arxiv.org/abs/2405.05386)  
**Date:** 2024-05-08  
**Summary page:** https://www.summarizepaper.com/en/paper/2405.05386v1/

## Key points

- Interpretability in artificial intelligence is crucial for explaining complex models to humans in understandable terms
- Two main paradigms of interpretability: intrinsic paradigm and post-hoc paradigm
- Debate between paradigms focuses on ensuring explanations are faithful to model behavior to prevent unwarranted confidence in AI systems
- Paper advocates for exploring new interpretability paradigms while maintaining a critical eye on faithfulness
- Introduces three emerging paradigms: measuring faithfulness during model design, optimizing models for faithful explanations, and proposing models capable of providing both predictions and explanations
- Caution needed when assessing the fidelity of explanations due to potential biases or inaccuracies in explanatory models
- Shift towards novel interpretability paradigms recommended to advance the field beyond existing limitations

## Summary

Interpretability in the field of artificial intelligence is crucial for explaining complex models to humans in understandable terms. Currently, interpretability is approached through two paradigms: the intrinsic paradigm and the post-hoc paradigm. The intrinsic paradigm focuses on designing models specifically for explanation while the post-hoc paradigm aims to explain black-box models. The debate between these paradigms centers around ensuring that explanations are faithful to the model's behavior to prevent unwarranted confidence in AI systems. This paper argues for exploring new paradigms in interpretability while maintaining a critical eye on faithfulness. Drawing parallels from the evolution of scientific paradigms, it suggests that constant innovation is necessary in this field. The paper introduces three emerging paradigms: one that emphasizes measuring faithfulness during model design, another that optimizes models for faithful explanations, and a third that proposes models capable of providing both predictions and explanations. While these new paradigms offer promising directions for interpretability, it is essential to remain cautious about claims of faithfulness. Past experiences have shown that visualizations and theoretical arguments can be misleading when assessing the fidelity of explanations. As researchers delve into new methods like learn-to-faithfully-explain approaches, they must be vigilant against unintentional biases or inaccuracies in their explanatory models. In conclusion, this paper advocates for a shift towards novel interpretability paradigms to advance the field beyond existing limitations. By challenging traditional beliefs and exploring innovative approaches, researchers can enhance our understanding of AI systems and ensure transparent communication between machines and humans.