# An Empirical Investigation of Catastrophic Forgeting in Gradient-Based Neural Networks

**Authors:** Ian J. Goodfellow, Mehdi Mirza, Xia Da, Aaron Courville, Yoshua Bengio  
**arXiv:** [1312.6211](https://arxiv.org/abs/1312.6211)  
**Date:** 2013-12-21  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/1312.6211v1/

## Key points


## Summary

In their paper titled "An Empirical Investigation of Catastrophic Forgetting in Gradient-Based Neural Networks," Ian J. Goodfellow, Mehdi Mirza, Xia Da, Aaron Courville, and Yoshua Bengio explore the issue of catastrophic forgetting faced by many machine learning models and algorithms. This phenomenon occurs when trained on one task and then trained on a second task; these models tend to forget how to perform the first task. This is particularly problematic for neural networks as they are prone to this type of forgetting. The authors investigate the extent to which this problem occurs in modern neural networks by comparing established and recent gradient-based training algorithms and activation functions. They also examine how the relationship between the first task and the second task affects catastrophic forgetting. Their empirical investigation provides insights into potential solutions for mitigating catastrophic forgetting in neural networks while shedding light on this important issue faced by machine learning models.