# Goal-Conditioned Reinforcement Learning: Problems and Solutions

**Authors:** Minghuan Liu, Menghui Zhu, Weinan Zhang  
**arXiv:** [2201.08299](https://arxiv.org/abs/2201.08299)  
**Date:** 2022-01-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2201.08299v1/

## Key points

- Goal-conditioned Reinforcement Learning (GCRL) is a subfield of reinforcement learning that trains agents to achieve different goals in complex scenarios.
- Unlike standard RL solutions, GCRL requires agents to make decisions based on specific goals.
- The survey provides a comprehensive overview of challenges and algorithms in GCRL.
- It addresses basic problems studied in GCRL and explores how agents can be trained to achieve various goals under specific scenarios.
- The authors discuss goal representation and explore methods such as high-level descriptions or explicit goal states.
- Challenges associated with goal specification are discussed and their impact on GCRL algorithm design is highlighted.
- Existing solutions for GCRL using intrinsic motivation, curiosity-driven exploration, or reward shaping techniques are examined.
- The strengths and limitations of these algorithms are analyzed by the authors.
- The survey concludes by discussing potential future prospects in GCRL research.

## Summary

Goal-conditioned Reinforcement Learning (GCRL) is a subfield of reinforcement learning (RL) that focuses on training agents to achieve different goals in complex scenarios. Unlike standard RL solutions that learn policies based solely on states or observations, GCRL requires agents to make decisions based on specific goals. In this survey, authors Minghuan Liu, Menghui Zhu, and Weinan Zhang provide a comprehensive overview of the challenges and algorithms in GCRL. The survey begins by addressing the basic problems studied in the field of GCRL. It explores how agents can be trained to achieve various goals under specific scenarios, highlighting the complexity involved. The authors then delve into goal representation and explain how existing solutions are designed from different perspectives. One key aspect discussed is how goals are represented within the RL framework. The authors explore various methods for goal representation such as using high-level descriptions or explicit goal states. They also discuss the challenges associated with goal specification and how it impacts the design of GCRL algorithms. The survey further examines existing solutions for GCRL from different points of view. It explores approaches that use intrinsic motivation, curiosity-driven exploration, or reward shaping techniques to guide agents towards achieving their goals. The authors analyze these algorithms and highlight their strengths and limitations. In conclusion, the survey provides insights into the challenges faced in GCRL and presents an overview of existing algorithms and approaches. Additionally, it discusses potential future prospects that recent research focuses on. This comprehensive analysis serves as a valuable resource for researchers and practitioners interested in understanding and advancing goal-conditioned reinforcement learning techniques.