# SkyEye: Self-Supervised Bird's-Eye-View Semantic Mapping Using Monocular Frontal View Images

**Authors:** Nikhil Gosala, Kürsat Petek, Paulo L. J. Drews-Jr, Wolfram Burgard, Abhinav Valada  
**arXiv:** [2302.04233](https://arxiv.org/abs/2302.04233)  
**Date:** 2023-02-08  
**Summary page:** https://www.summarizepaper.com/en/paper/2302.04233v1/

## Key points

- The paper addresses the need for Bird's-Eye-View (BEV) semantic maps in automated driving pipelines.
- Existing approaches for generating BEV maps rely on fully supervised training and require large amounts of annotated data.
- The authors propose a self-supervised approach for generating a BEV semantic map using a single monocular image from the frontal view (FV).
- The model leverages FV semantic annotations from video sequences during training instead of BEV ground truth annotations.
- The proposed SkyEye architecture learns through implicit supervision and explicit supervision.
- Extensive evaluations on the KITTI-360 dataset show that the self-supervised approach performs comparably to state-of-the-art fully supervised methods.
- It achieves competitive results using only 1% of direct supervision in the BEV compared to fully supervised approaches.
- The authors publicly release their code and the BEV datasets generated from the KITTI-360 and Waymo datasets to facilitate further research.

## Summary

The paper titled "SkyEye: Self-Supervised Bird's-Eye-View Semantic Mapping Using Monocular Frontal View Images" addresses the need for Bird's-Eye-View (BEV) semantic maps in automated driving pipelines. These maps provide a rich representation that is crucial for decision-making tasks. However, existing approaches for generating BEV maps rely on fully supervised training and require large amounts of annotated data. To overcome this limitation, the authors propose the first self-supervised approach for generating a BEV semantic map using a single monocular image from the frontal view (FV). Instead of relying on BEV ground truth annotations, the model leverages more easily available FV semantic annotations from video sequences during training. The proposed SkyEye architecture learns through two modes of self-supervision: implicit supervision and explicit supervision. Implicit supervision enforces spatial consistency of the scene over time based on FV semantic sequences. Explicit supervision utilizes BEV pseudolabels generated from FV semantic annotations and self-supervised depth estimates. Extensive evaluations on the KITTI-360 dataset demonstrate that the self-supervised approach performs comparably to state-of-the-art fully supervised methods. Remarkably, it achieves competitive results using only 1% of direct supervision in the BEV compared to fully supervised approaches. In addition to presenting their approach, the authors publicly release both their code and the BEV datasets generated from the KITTI-360 and Waymo datasets to facilitate further research in this area. Overall, this paper introduces an innovative self-supervised approach which reduces reliance on annotated data while maintaining performance levels comparable to fully supervised methods.