# Pearl: A Production-ready Reinforcement Learning Agent

**Authors:** Zheqing Zhu, Rodrigo de Salvo Braz, Jalaj Bhandari, Daniel Jiang, Yi Wan, Yonathan Efroni, Liyuan Wang, Ruiyang Xu, Hongbo Guo, Alex Nikulkov, Dmytro Korenkevych, Urun Dogan, Frank Cheng, Zheng Wu, Wanqiao Xu  
**arXiv:** [2312.03814](https://arxiv.org/abs/2312.03814)  
**Date:** 2023-12-06  
**Summary page:** https://www.summarizepaper.com/en/paper/2312.03814v1/

## Key points

- Introduction to Pearl, a production-ready RL agent software package
- Motivation behind Pearl's development
- Key features and design choices of Pearl
- Simple illustrations of Pearl's user interface
- Comparison of Pearl to other open-source RL libraries
- Initial benchmarking results of Pearl
- Current industry adoptions of Pearl to demonstrate its readiness for production usage
- Detailed description of the design of PearlAgent, including its five main modules: policy_learner, exploration_module, history_summarization_module, safety_module, and replay_buffer
- Introduction of notations that will be used throughout the paper to aid in understanding the modules
- Prioritization of key elements in the agent's design for efficient learning in practical sequential decision-making problems, including offline learning/pretraining, online learning with exploration capabilities, and safe learning with the ability to incorporate safety or preference constraints.

## Summary

This paper serves as an introduction to Pearl, a production-ready reinforcement learning (RL) agent software package. It discusses the motivation behind Pearl's development and its key features and design choices. Additionally, it provides simple illustrations of its user interface and compares it to other open-source RL libraries. The paper also presents initial benchmarking results and highlights current industry adoptions of Pearl to demonstrate its readiness for production usage. In Section 2, the design of PearlAgent is described in detail. This includes an overview of its five main modules: policy_learner, exploration_module, history_summarization_module, safety_module, and replay_buffer. To aid in understanding these modules, the paper introduces notations that will be used throughout the rest of the paper. The agent's design prioritizes several key elements essential for efficient learning in practical sequential decision-making problems. These elements include offline learning/pretraining, online learning with exploration capabilities, and safe learning with the ability to incorporate safety or preference constraints.