# Automated Reinforcement Learning (AutoRL): A Survey and Open Problems

**Authors:** Jack Parker-Holder, Raghu Rajan, Xingyou Song, André Biedenkapp, Yingjie Miao, Theresa Eimer, Baohe Zhang, Vu Nguyen, Roberto Calandra, Aleksandra Faust, Frank Hutter, Marius Lindauer  
**arXiv:** [2201.03916](https://arxiv.org/abs/2201.03916)  
**Date:** 2022-01-11  
**Summary page:** https://www.summarizepaper.com/en/paper/2201.03916v1/

## Key points

- Automated Reinforcement Learning (AutoRL) combines RL with deep learning to develop generally capable agents
- RL agents heavily rely on manual tuning and design choices in the training process
- AutoML automates design choices in other areas of machine learning and has shown promising results when applied to RL
- AutoRL presents unique challenges that require a different set of methods compared to standard applications of AutoML
- AutoRL has demonstrated promise in various applications such as RNA design and playing games like Go
- Research on AutoRL has been conducted in distinct subfields ranging from meta-learning to evolution
- This survey provides a common taxonomy for the field of AutoRL and discusses each area in detail
- The survey poses open problems that would be of interest to researchers moving forward.

## Summary

Automated Reinforcement Learning (AutoRL) is an emerging field of research that combines Reinforcement Learning (RL) with deep learning to develop generally capable agents. RL has shown impressive feats when combined with deep learning, but the success of RL agents heavily relies on manual tuning and design choices in the training process. This makes it challenging to apply RL to new problems and limits its full potential. AutoML, which automates design choices in other areas of machine learning, has shown promising results when applied to RL. However, AutoRL presents unique challenges that require a different set of methods compared to standard applications of AutoML. These challenges arise from the nature of RL itself. Despite these challenges, AutoRL has demonstrated promise in various applications such as RNA design and playing games like Go. Due to the diversity of methods and environments considered in RL, research on AutoRL has been conducted in distinct subfields ranging from meta-learning to evolution. To unify the field of AutoRL, this survey provides a common taxonomy and discusses each area in detail. Additionally, the survey poses open problems that would be of interest to researchers moving forward.