# An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling

**Authors:** Shaojie Bai, J. Zico Kolter, Vladlen Koltun  
**arXiv:** [1803.01271](https://arxiv.org/abs/1803.01271)  
**Date:** 2018-03-04  
**Summary page:** https://www.summarizepaper.com/en/paper/1803.01271v1/

## Key points

- Authors challenge the common association between sequence modeling and recurrent networks
- Recent findings suggest convolutional architectures may outperform recurrent networks in tasks like audio synthesis and machine translation
- Systematic evaluation showed simple convolutional architecture consistently outperformed canonical recurrent networks like LSTMs on various tasks and datasets
- Convolutional networks demonstrated longer effective memory compared to recurrent networks
- Authors advocate for reconsidering traditional preference for recurrent networks in sequence modeling tasks
- Propose that convolutional networks should be viewed as a natural starting point due to superior performance across applications

## Summary

In their paper titled "An Empirical Evaluation of Generic Convolutional and Recurrent Networks for Sequence Modeling," authors Shaojie Bai, J. Zico Kolter, and Vladlen Koltun challenge the common association between sequence modeling and recurrent networks. While recurrent networks have long been synonymous with sequence modeling in the deep learning community, recent findings suggest that convolutional architectures may actually outperform recurrent networks on tasks such as audio synthesis and machine translation. To address this discrepancy, the authors conducted a systematic evaluation of generic convolutional and recurrent architectures for sequence modeling across a diverse range of standard tasks commonly used to benchmark recurrent networks. Surprisingly, their results revealed that a simple convolutional architecture consistently outperformed canonical recurrent networks like LSTMs on various tasks and datasets while also demonstrating longer effective memory. Based on their findings, the authors advocate for reconsidering the traditional preference for recurrent networks in sequence modeling tasks. Instead, they propose that convolutional networks should be viewed as a natural starting point for such tasks due to their superior performance across a broad spectrum of applications. This study sheds new light on the potential of convolutional architectures in sequence modeling and challenges existing assumptions within the deep learning community regarding the most effective approach to tackling these types of tasks.