# Prioritized Experience Replay

**Authors:** Tom Schaul, John Quan, Ioannis Antonoglou, David Silver  
**arXiv:** [1511.05952](https://arxiv.org/abs/1511.05952)  
**Date:** 2015-11-18  
**Summary page:** https://www.summarizepaper.com/en/paper/1511.05952v4/

## Key points

- Authors introduce a novel approach to experience replay in reinforcement learning
- Technique allows online agents to remember and reuse past experiences, enhancing learning efficiency
- Framework proposed for prioritizing experiences based on their significance
- Replay important transitions more frequently for more effective learning
- Application of prioritized experience replay to Deep Q-Networks (DQN)
- DQN with prioritized experience replay outperforms DQN with uniform replay on 41 out of 49 games
- Establishes a new state-of-the-art benchmark in reinforcement learning performance
- Importance of prioritizing experiences highlighted for improving learning capabilities in complex tasks like playing Atari games

## Summary

In their paper titled "Prioritized Experience Replay," authors Tom Schaul, John Quan, Ioannis Antonoglou, and David Silver introduce a novel approach to experience replay in reinforcement learning. This technique allows online agents to remember and reuse past experiences, enhancing learning efficiency. The traditional method involved uniformly sampling experience transitions from a replay memory without considering their importance. However, the authors propose a framework for prioritizing experiences based on their significance. By replaying important transitions more frequently, the agents can learn more effectively. The authors apply this prioritized experience replay technique to Deep Q-Networks (DQN), a popular reinforcement learning algorithm known for achieving human-level performance in various Atari games. Through experiments, they demonstrate that DQN with prioritized experience replay surpasses the performance of DQN with uniform replay on 41 out of 49 games, establishing a new state-of-the-art benchmark. This research published at ICLR 2016 sheds light on the importance of prioritizing experiences in reinforcement learning algorithms like DQN. By focusing on significant transitions during training, agents can improve their learning capabilities and achieve superior performance in complex tasks such as playing Atari games. The findings highlight the potential impact of prioritized experience replay in advancing the field of reinforcement learning and artificial intelligence as a whole.