# State2Explanation: Concept-Based Explanations to Benefit Agent Learning and User Understanding

**Authors:** Devleena Das, Sonia Chernova, Been Kim  
**arXiv:** [2309.12482](https://arxiv.org/abs/2309.12482)  
**Date:** 2023-09-21  
**Summary page:** https://www.summarizepaper.com/en/paper/2309.12482v1/

## Key points

- The paper addresses the need for explanations of AI decision making that are understandable by non-experts.
- Existing methods for generating concept-based explanations focus on classification techniques and lack applicability to sequential decision making scenarios.
- The authors propose a framework called State2Explanation (S2E) that aims to provide concept-based explanations in sequential decision making settings.
- The S2E framework involves learning a joint embedding model between state-action pairs and concept-based explanations.
- The framework informs reward shaping during an agent's training process and provides explanations to end-users at deployment time for improved task performance.
- Experimental evaluations using Connect 4 and Lunar Lander environments validate the success of S2E in improving agent learning rate and end-user task performance.
- The State2Explanation framework bridges the gap between complex AI systems and non-expert users through interpretable explanations.

## Summary

The paper titled "State2Explanation: Concept-Based Explanations to Benefit Agent Learning and User Understanding" by Devleena Das, Sonia Chernova, and Been Kim addresses the need for explanations of AI decision making that are understandable by non-experts. As AI systems become more complex and widely used in daily tasks, there is a growing effort to develop methods that can provide explanations to users who may not have expertise in AI. One popular approach to generating explanations is leveraging higher-level concepts and producing concept-based explanations. However, most existing methods focus on classification techniques and lack applicability to sequential decision making scenarios. The authors argue that there is a limited scope of concept-based explanations for sequential decision making. To address this gap, the authors propose a framework called State2Explanation (S2E) that aims to provide concept-based explanations in sequential decision making settings. They first define desiderata for defining "concepts" in such settings. Additionally, they draw inspiration from the Protege Effect, which suggests that explaining knowledge reinforces one's self-learning. The S2E framework involves learning a joint embedding model between state-action pairs and concept-based explanations. This learned model serves two purposes: (1) informing reward shaping during an agent's training process and (2) providing explanations to end-users at deployment time for improved task performance. The authors validate their framework through experimental evaluations using Connect 4 and Lunar Lander environments. The results demonstrate the success of S2E in providing a dual benefit - it informs reward shaping during training, leading to improved agent learning rate, while also significantly improving end-user task performance by providing understandable explanations. In conclusion, the paper presents the State2Explanation framework as an effective method for generating concept-based explanations in sequential decision making scenarios. The framework not only enhances agent learning but also improves user understanding of AI decision making by bridging the gap between complex AI systems and non-expert users through interpretable explanations.