# Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progress

**Authors:** Renjie Wu, Eamonn J. Keogh  
**arXiv:** [2009.13807](https://arxiv.org/abs/2009.13807)  
**Date:** 2020-09-29  
**Summary page:** https://www.summarizepaper.com/en/paper/2009.13807v5/

## Key points

- Time series anomaly detection is an important topic in data science
- Recent interest has increased due to the success of deep learning
- Most papers in this area rely on popular benchmark datasets created by organizations such as Yahoo, Numenta, and NASA
- Majority of individual exemplars within these datasets suffer from four flaws
- These flaws raise concerns about the reliability of published comparisons between anomaly detection algorithms and question the true progress made in recent years
- Wu and Keogh introduce the UCR Time Series Anomaly Archive as a resource to overcome these limitations
- The archive will serve a similar role to the UCR Time Series Classification Archive by providing a benchmark for meaningful comparisons between different approaches and offering an accurate measure of overall progress in time series anomaly detection
- This paper sheds light on existing flaws present in current time series anomaly detection benchmarks and emphasizes the need for more reliable evaluation methods

## Summary

In their paper titled "Current Time Series Anomaly Detection Benchmarks are Flawed and are Creating the Illusion of Progress," authors Renjie Wu and Eamonn J. Keogh discuss the importance of time series anomaly detection in data science. They highlight that this field has been a significant topic since the 1950s, but recent interest has surged due to the success of deep learning in various domains and time series tasks. The authors point out that most papers in this area rely on a few popular benchmark datasets created by organizations such as Yahoo, Numenta, and NASA. However, they make a surprising claim that the majority of individual exemplars within these datasets suffer from four flaws which raise concerns about the reliability of published comparisons between anomaly detection algorithms and question the true progress made in recent years. To address these issues, Wu and Keogh introduce the UCR Time Series Anomaly Archive as a resource to overcome these limitations. This archive will serve a similar role to the UCR Time Series Classification Archive by providing a benchmark for meaningful comparisons between different approaches and offering an accurate measure of overall progress in time series anomaly detection. Overall, this paper sheds light on existing flaws present in current time series anomaly detection benchmarks and emphasizes the need for more reliable evaluation methods while introducing the UCR Time Series Anomaly Archive to facilitate advancements in this field.