# In deep reinforcement learning, a pruned network is a good network

**Authors:** Johan Obando-Ceron, Aaron Courville, Pablo Samuel Castro  
**arXiv:** [2402.12479](https://arxiv.org/abs/2402.12479)  
**Date:** 2024-02-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2402.12479v1/

## Key points

- Study by Johan Obando-Ceron, Aaron Courville, and Pablo Samuel Castro focuses on challenges faced by deep reinforcement learning agents in utilizing network parameters effectively.
- Gradual magnitude pruning enhances parameter effectiveness and leads to networks that outperform traditional ones.
- Pruned networks achieve remarkable performance with only a fraction of full network parameters due to the "scaling law" phenomenon.
- Focus on four games - BeamRider, Breakout, Enduro, and VideoPinball - with analyses on Q estimates variance, network parameters norm, Q-values norm, effective rank of the matrix, and fraction of dormant neurons.
- Pruning reduces variance and norms of parameters while increasing the effective rank due to normalization effects and increased network plasticity.
- Comparison against weight decay (WD) and L2 regularization highlights unique benefits of gradual magnitude pruning in enhancing parameter effectiveness and improving overall network performance.
- Importance of exploring alternative pruning schedules for further optimization in deep reinforcement learning networks is emphasized.
- Leveraging sparse training methods like gradual magnitude pruning can lead to significant advancements in network efficiency and performance across various tasks and environments.

## Summary

The study by Johan Obando-Ceron, Aaron Courville, and Pablo Samuel Castro delves into the challenges faced by deep reinforcement learning agents in effectively utilizing their network parameters. Through gradual magnitude pruning, the researchers demonstrate how this approach can greatly enhance parameter effectiveness and lead to networks that outperform traditional ones. This unique "scaling law" phenomenon is observed as these pruned networks achieve remarkable performance with only a fraction of the full network parameters. The study focuses on four different games - BeamRider, Breakout, Enduro, and VideoPinball - and conducts various analyses to measure the impact of pruning on Q estimates variance, network parameters norm, Q-values norm, effective rank of the matrix, and fraction of dormant neurons. The results show that pruning reduces variance and norms of parameters while increasing the effective rank of parameters due to normalization effects and increased network plasticity. By comparing their method against existing techniques like weight decay (WD) and L2 regularization, the researchers highlight the unique benefits offered by gradual magnitude pruning in enhancing parameter effectiveness and improving overall network performance. This study emphasizes the importance of exploring alternative pruning schedules for further optimization in deep reinforcement learning networks. It also suggests that leveraging sparse training methods such as gradual magnitude pruning can lead to significant advancements in network efficiency and performance across various tasks and environments.