# StreamYOLO: Real-time Object Detection for Streaming Perception

**Authors:** Jinrong Yang, Songtao Liu, Zeming Li, Xiaoping Li, Jian Sun  
**arXiv:** [2207.10433](https://arxiv.org/abs/2207.10433)  
**Date:** 2022-07-21  
**Summary page:** https://www.summarizepaper.com/en/paper/2207.10433v1/

## Key points

- The paper addresses the need for fast and accurate inference in perceptive models for autonomous driving
- Existing works often overlook environmental changes after processing, leading to potential safety issues
- Introduces the concept of streaming perception, combining latency and accuracy into a single metric for online video perception
- Evaluates real-time models based on this metric and enhances their capabilities by incorporating predictive abilities
- Proposes a framework consisting of the Dual Flow Perception module (DFP) and Trend Aware Loss (TAL)
- DFP module captures dynamic flow and static flow in parallel to capture moving tendencies and basic detection features
- TAL adaptively generates loss weights for each object based on its moving speed
- Considers multiple velocities in driving scenes and introduces Velocity-aware streaming AP (VsAP) as a joint evaluation metric for accuracy
- Designs an efficient mix-velocity training strategy to enable detectors to perceive any velocities effectively
- Achieves state-of-the-art performance on the Argoverse-HD dataset, improving both average precision (sAP) and VsAP by 4.7% and 8.2%, respectively
- Contributes significantly to advancing autonomous driving technology by addressing challenges associated with fast inference and environmental changes during online perception tasks

## Summary

The paper titled "StreamYOLO: Real-time Object Detection for Streaming Perception" addresses the need for fast and accurate inference in perceptive models for autonomous driving. The authors highlight that existing works often overlook the environmental changes that occur after processing, leading to potential safety issues. To overcome this limitation, the concept of streaming perception is introduced, which combines latency and accuracy into a single metric for online video perception. In this study, the authors focus on evaluating the performance of real-time models based on this metric and enhancing their capabilities by incorporating predictive abilities. They propose a simple framework consisting of two modules: the Dual Flow Perception module (DFP) and Trend Aware Loss (TAL). The DFP module captures both dynamic flow and static flow in parallel to capture moving tendencies and basic detection features, respectively. On the other hand, TAL adaptively generates loss weights for each object based on its moving speed. To make their approach more realistic, the authors consider multiple velocities in driving scenes and introduce Velocity-aware streaming AP (VsAP) as a joint evaluation metric for accuracy. They also design an efficient mix-velocity training strategy to enable detectors to perceive any velocities effectively. The results demonstrate that the proposed method achieves state-of-the-art performance on the Argoverse-HD dataset. Compared to a strong baseline, it improves both average precision (sAP) and VsAP by 4.7% and 8.2%, respectively, validating its effectiveness in real-time object detection for streaming perception. Overall, this paper contributes significantly to advancing autonomous driving technology by addressing challenges associated with fast inference and environmental changes during online perception tasks. The proposed framework demonstrates promising results in terms of accuracy and speed tradeoffs while considering future predictions.