# Motion Forecasting in Continuous Driving

**Authors:** Nan Song, Bozhou Zhang, Xiatian Zhu, Li Zhang  
**arXiv:** [2410.06007](https://arxiv.org/abs/2410.06007)  
**Date:** 2024-10-08  
**Summary page:** https://www.summarizepaper.com/en/paper/2410.06007v1/

## Key points

- Motion forecasting for agents in autonomous driving is complex and challenging, requiring consideration of numerous possibilities for each agent's next action and their interactions in space and time.
- Existing methods often overlook situational and contextual relationships between successive driving scenes, leading to suboptimal solutions that are inefficient in practice.
- RealMotion framework consists of two integral streams: the scene context stream and the agent trajectory stream, which capture temporal interactive relationships among scene elements and optimize current forecasting by relaying past predictions sequentially.
- A data reorganization strategy has been introduced to bridge the gap between existing benchmarks and real-world applications, enabling a broader exploitation of situational insights into dynamic motion across space and time.
- RealMotion achieves state-of-the-art performance with efficient real-world inference capabilities, supporting successive forecasting actions over space and time through its components.

## Summary

, , , , 

Motion forecasting for agents in autonomous driving is a complex and challenging task that requires considering numerous possibilities for each agent's next action and their interactions in space and time. In real-world applications, motion forecasting must occur continuously as the self-driving car moves. However, existing methods often overlook situational and contextual relationships between successive driving scenes by processing each scene independently within a certain range. This simplification leads to suboptimal solutions that are inefficient in practice. To overcome this limitation, a novel framework called RealMotion has been proposed. RealMotion consists of two integral streams at the scene level: the scene context stream and the agent trajectory stream. The scene context stream accumulates historical information progressively until the present moment to capture temporal interactive relationships among scene elements. Meanwhile, the agent trajectory stream optimizes current forecasting by sequentially relaying past predictions. Additionally, a data reorganization strategy has been introduced to bridge the gap between existing benchmarks and real-world applications, aligning with the network's design. These approaches enable a broader exploitation of situational and progressive insights into dynamic motion across space and time. Extensive experiments conducted on Argoverse series with various settings have demonstrated that RealMotion achieves state-of-the-art performance while also offering efficient real-world inference capabilities. In conclusion, this work aims to address motion forecasting from a practical continuous driving perspective by placing it within a wider scene context compared to previous approaches. RealMotion serves as a generic framework specifically designed to support successive forecasting actions over space and time through its scene context stream and agent trajectory stream components. The sequential nature of these components allows for progressive capture of critical information essential for accurate motion forecasting in autonomous driving scenarios. Furthermore, this research has been accepted at NeurIPS 2024 Spotlight conference, showcasing its significance in advancing the field of autonomous driving technology. The source code for RealMotion is available at https://github.com/fudan-zvg/RealMotion for further exploration and implementation by interested parties.