# aiMotive Dataset: A Multimodal Dataset for Robust Autonomous Driving with Long-Range Perception

**Authors:** Tamás Matuszka, Iván Barton, Ádám Butykai, Péter Hajas, Dávid Kiss, Domonkos Kovács, Sándor Kunsági-Máté, Péter Lengyel, Gábor Németh, Levente Pető, Dezső Ribli, Dávid Szeghy, Szabolcs Vajna, Bálint Varga  
**arXiv:** [2211.09445](https://arxiv.org/abs/2211.09445)  
**Date:** 2022-11-17  
**Summary page:** https://www.summarizepaper.com/en/paper/2211.09445v3/

## Key points

- Autonomous driving is a rapidly growing research area in computer vision
- Robustness is crucial for real-world deployment of autonomous vehicles
- A multimodal dataset for robust autonomous driving with long-range perception capabilities has been introduced
- The dataset consists of 176 scenes captured in different environments and under varying weather conditions
- Data from LiDAR, camera, and radar sensors covering a 360-degree field of view is included in the dataset
- The dataset is annotated with 3D bounding boxes and provides consistent identifiers across frames
- Tasks that can be performed using the dataset include 3D object detection, end-to-end long-range multiple object tracking, and motion prediction
- The dataset also includes high-quality GNSS-INS sensory data for training odometry algorithms
- It can be used for contrastive representation learning by learning similar representations for different sensor modalities corresponding to the same frame
- Unimodal and multimodal baseline models have been developed and compared on the dataset
- Limitations include synchronization discrepancies between sensors and the lack of side radars

## Summary

Autonomous driving is a rapidly growing research area in computer vision. It has the potential to revolutionize transportation and make roads safer for everyone. However, ensuring robustness is crucial for real-world deployment of autonomous vehicles. This requires datasets that are diverse and comprehensive enough to handle various weather conditions and long-range perception. To address this need, we introduce a multimodal dataset for robust autonomous driving with long-range perception capabilities. Our dataset consists of 176 scenes captured in different environments and under varying weather conditions such as rain. It includes synchronized and calibrated data from LiDAR, camera, and radar sensors covering a 360-degree field of view. The dataset is annotated with 3D bounding boxes that have consistent identifiers across frames. In addition to 3D object detection, our dataset can be used for various tasks such as end-to-end long-range multiple object tracking using unique track IDs provided in the dataset. We also propose motion prediction as another task that can be performed using our dataset. This is essential for functions like Automatic Emergency Braking or Adaptive Cruise Control in autonomous driving systems. Our dataset also includes high-quality GNSS-INS sensory data that enables training and benchmarking of various odometry algorithms. Furthermore, it can be used for contrastive representation learning by learning similar representations for different sensor modalities corresponding to the same frame in a self-supervised manner. Overall, our diverse multimodal dataset provides robust long-range perception capabilities through redundant sensor coverage. We have developed unimodal and multimodal baseline models and compared their performance on the dataset. While there are limitations such as synchronization discrepancies between sensors and the lack of side radars, we aim to extend the dataset with additional environmental and weather conditions in future work. We believe that our   will be valuable to the research community, allowing them to build upon our baselines and significantly improve the performance of . The dataset is available at \url{https://github.com/aimotive/aimotive_dataset}.