# Multi-Camera Calibration Free BEV Representation for 3D Object Detection

**Authors:** Hongxiang Jiang, Wenming Meng, Hongmei Zhu, Qian Zhang, Jihao Yin  
**arXiv:** [2210.17252](https://arxiv.org/abs/2210.17252)  
**Date:** 2022-10-31  
**Summary page:** https://www.summarizepaper.com/en/paper/2210.17252v1/

## Key points

- Learning a Bird's Eye View (BEV) representation is crucial for autonomous driving
- Existing methods relying on depth estimation or camera-driven attention are not stable under noisy camera parameters
- Challenges: accurate depth prediction and calibration
- Introducing Multi-Camera Calibration Free Transformer (CFT) for robust BEV representation
- CFT focuses on exploring implicit mapping instead of relying on camera intrinsics and extrinsics
- CFT uses position-aware enhancement (PA) technique to mine potential 3D information in BEV
- CFT proposes view-aware attention mechanism for more effective interaction and reduced computation
- Impressive performance on nuScenes detection task leaderboard with NDS score of 49.7%
- Comparable to other geometry-guided methods without relying on camera parameters
- Achieves high performance without requiring temporal input or other modal information
- View-attention variant reduces memory usage and transformer FLOPs by approximately 12% and 60%, respectively, while improving NDS score by 1.0%
- CFT is naturally robust to noisy camera parameters, giving it a competitive advantage over existing methods
- Novel approach for achieving robust BEV representation in autonomous driving applications

## Summary

In the field of autonomous driving, learning a Bird's Eye View (BEV) representation from surrounding views is crucial for a multi-task framework. However, existing methods that rely on depth estimation or camera-driven attention are not stable when it comes to obtaining accurate transformations under noisy camera parameters. This instability arises from two main challenges: accurate depth prediction and calibration. To address these challenges, this work introduces a completely Multi-Camera Calibration Free Transformer (CFT) for robust BEV representation. Unlike previous approaches that rely on camera intrinsics and extrinsics, CFT focuses on exploring implicit mapping. It achieves this by mining potential 3D information in BEV through a position-aware enhancement (PA) technique, which guides better feature learning from image views to BEV. Instead of using camera-driven point-wise or global transformations, CFT proposes a view-aware attention mechanism. This attention mechanism allows for interaction within more effective regions while reducing redundant computation and promoting convergence. As a result, CFT achieves impressive performance on the nuScenes detection task leaderboard with an NDS score of 49.7%. Importantly, it is the first work to remove the reliance on camera parameters while still being comparable to other geometry-guided methods. Notably, CFT achieves its high performance without requiring temporal input or other modal information. Furthermore, it achieves the second-highest performance with a smaller image input size of 1600 * 640. This is made possible by the view-attention variant of CFT, which reduces memory usage and transformer FLOPs (floating-point operations) by approximately 12% and 60%, respectively. Additionally, this variant improves the NDS score by 1.0%. One key advantage of CFT is its natural robustness to noisy camera parameters. This robustness makes CFT more competitive compared to existing methods that struggle with such noise. Overall, this work presents a novel approach for achieving robust BEV representation in autonomous driving applications. The proposed CFT method overcomes the challenges of accurate depth prediction and calibration by leveraging implicit mapping and view-aware attention.