# Causal Reinforcement Learning: A Survey

**Authors:** Zhihong Deng, Jing Jiang, Guodong Long, Chengqi Zhang  
**arXiv:** [2307.01452](https://arxiv.org/abs/2307.01452)  
**Date:** 2023-07-04  
**Summary page:** https://www.summarizepaper.com/en/paper/2307.01452v1/

## Key points

- Reinforcement learning is a crucial paradigm for addressing sequential decision problems amidst uncertainty.
- Practical application of reinforcement learning techniques remains complex due to agents lacking inherent understanding of their environment and struggling with explanations and knowledge generalization.
- Causality is highlighted as a powerful tool for formalizing knowledge systematically and facilitating effective knowledge transfer in reinforcement learning.
- Causal reinforcement learning is a specialized subfield aiming to enhance existing algorithms by incorporating causal relationships into the learning process.
- The survey by Deng et al. comprehensively reviews existing literature on causal reinforcement learning, categorizing methodologies and analyzing target problems and specific techniques employed.
- Open challenges and future directions within causal reinforcement learning are outlined, providing insights for further research and development in the field.

## Summary

In their paper titled "Causal Reinforcement Learning: A Survey," authors Zhihong Deng, Jing Jiang, Guodong Long, and Chengqi Zhang delve into the realm of reinforcement learning, a crucial paradigm for tackling sequential decision problems amidst uncertainty. Despite significant advancements in recent years, the practical application of reinforcement learning techniques remains complex. One of the primary challenges lies in the fact that reinforcement learning agents lack an inherent understanding of their environment, necessitating them to learn from scratch through repeated trial-and-error interactions. Furthermore, these agents often struggle to provide explanations for their decisions and generalize the knowledge they acquire. To address these challenges, the authors highlight the significance of causality as a powerful tool for formalizing knowledge systematically and leveraging invariance to facilitate effective knowledge transfer. This has paved the way for the emergence of causal reinforcement learning, a specialized subfield within reinforcement learning that aims to enhance existing algorithms by incorporating causal relationships into the learning process. The survey conducted by Deng et al. comprehensively reviews existing literature on causal reinforcement learning. The authors begin by introducing fundamental concepts related to causality and reinforcement learning before delving into how causality can effectively tackle core issues encountered in non-causal reinforcement learning approaches. They categorize and systematically analyze various existing methodologies within causal reinforcement learning based on their target problems and specific techniques employed. In conclusion, Deng et al. outline open challenges and future directions within this burgeoning field, shedding light on potential avenues for further research and development in causal reinforcement learning. With their thorough examination of this subject matter, the authors contribute valuable insights that could potentially shape the future trajectory of reinforcement learning applications in real-world scenarios.