# Mastering Atari with Discrete World Models

**Authors:** Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, Jimmy Ba  
**arXiv:** [2010.02193](https://arxiv.org/abs/2010.02193)  
**Date:** 2020-10-05  
**Summary page:** https://www.summarizepaper.com/en/paper/2010.02193v1/

## Key points

- DreamerV2 achieves human-level performance on the Atari benchmark of 55 tasks
- World models are crucial for generalization in intelligent agents by allowing them to learn from imagined outcomes
- DreamerV2 learns behaviors purely from predictions within a compact latent space of a powerful world model
- The world model in DreamerV2 uses discrete representations and is trained separately from the policy, marking a departure from traditional approaches
- DreamerV2 surpasses other top single-GPU agents IQN and Rainbow in final performance metrics by reaching 200 million frames with the same computational budget and wall-clock time
- The success of DreamerV2 demonstrates the efficacy of leveraging world models for behavior learning and showcases potential for advancing reinforcement learning techniques

## Summary

In their paper titled "Mastering Atari with Discrete World Models," authors Danijar Hafner, Timothy Lillicrap, Mohammad Norouzi, and Jimmy Ba introduce DreamerV2: a reinforcement learning agent that achieves human-level performance on the Atari benchmark of 55 tasks. The key challenge in developing intelligent agents lies in their ability to generalize from past experiences to achieve goals in complex environments. World models play a crucial role in facilitating this generalization by allowing agents to learn behaviors from imagined outcomes, thereby increasing sample-efficiency. While the concept of learning world models from image inputs has gained traction for certain tasks, accurately modeling Atari games to derive successful behaviors has remained a longstanding open challenge. DreamerV2 addresses this challenge by learning behaviors purely from predictions within the compact latent space of a powerful world model. This world model utilizes discrete representations and is trained separately from the policy, marking a significant departure from traditional approaches. The groundbreaking aspect of DreamerV2 lies in its achievement of human-level performance on the Atari benchmark through learning behaviors inside a separately trained world model. With the same computational budget and wall-clock time as other top single-GPU agents IQN and Rainbow, DreamerV2 surpasses their final performance metrics by reaching 200 million frames. This innovative approach not only demonstrates the efficacy of leveraging world models for behavior learning but also showcases the potential for advancing reinforcement learning techniques in complex environments such as Atari games. The success of DreamerV2 underscores the importance of exploring novel strategies for training intelligent agents and opens up new possibilities for achieving high-performance outcomes in challenging domains.