# RoScenes: A Large-scale Multi-view 3D Dataset for Roadside Perception

**Authors:** Xiaosu Zhu, Hualian Sheng, Sijia Cai, Bing Deng, Shaopeng Yang, Qiao Liang, Ken Chen, Lianli Gao, Jingkuan Song, Jieping Ye  
**arXiv:** [2405.09883](https://arxiv.org/abs/2405.09883)  
**Date:** 2024-05-16  
**Summary page:** https://www.summarizepaper.com/en/paper/2405.09883v1/

## Key points

- RoScenes is the largest multi-view roadside perception dataset designed to advance Bird's Eye View (BEV) approaches for complex traffic scenes.
- It features expansive perception area, comprehensive scene coverage, and dense traffic scenarios.
- Contains 21.13 million 3D annotations within a compact 64,000 $m^2$ area.
- Utilizes a novel BEV-to-3D joint annotation pipeline to efficiently gather data while addressing challenges of costly roadside 3D labeling.
- Current BEV methods evaluated on RoScenes show limitations in handling extensive perception areas and diverse sensor layouts across scenes, leading to subpar performance levels.
- RoBEV method proposed with feature-guided position embedding for effective 2D-3D feature assignment surpasses existing state-of-the-art methods without additional computational overhead on the validation set.
- Detailed statistics and analysis include camera parameters such as occlusion levels, focal length, pitch angle, mounting height, and road coverage.
- Refined BEV annotations implemented to mitigate perspective distortions and jittering effects from UAV imagery.

## Summary

RoScenes is the largest multi-view roadside perception dataset designed to advance Bird's Eye View (BEV) approaches for complex traffic scenes. It stands out for its expansive perception area, comprehensive scene coverage, and dense traffic scenarios. With an impressive 21.13 million 3D annotations within a compact 64,000 $m^2$ area, RoScenes utilizes a novel BEV-to-3D joint annotation pipeline to efficiently gather this vast amount of data while addressing the challenges of costly roadside 3D labeling. A thorough evaluation of current BEV methods on RoScenes reveals limitations in handling the extensive perception area and diverse sensor layouts across scenes, leading to subpar performance levels. In response, the RoBEV method is proposed with feature-guided position embedding for effective 2D-3D feature assignment. This approach surpasses existing state-of-the-art methods without additional computational overhead on the validation set. The dataset includes detailed statistics and analysis showcasing camera parameters such as occlusion levels, focal length, pitch angle, mounting height, and road coverage. Additionally, refined BEV annotations are implemented to mitigate perspective distortions and jittering effects from UAV imagery.