# Unsupervised OmniMVS: Efficient Omnidirectional Depth Inference via Establishing Pseudo-Stereo Supervision

**Authors:** Zisong Chen, Chunyu Lin, Nie Lang, Kang Liao, Yao Zhao  
**arXiv:** [2302.09922](https://arxiv.org/abs/2302.09922)  
**Date:** 2023-02-20  
**Summary page:** https://www.summarizepaper.com/en/paper/2302.09922v1/

## Key points

- Omnidirectional multi-view stereo (MVS) vision provides ultra-wide field-of-view for perceiving 360-degree 3D surroundings
- Traditional solutions in this domain require costly dense depth labels, making them impractical for real-world applications
- A groundbreaking approach is introduced: the first unsupervised omnidirectional MVS framework based on multiple fisheye images
- The innovative framework involves projecting input images to a virtual view center and creating panoramic images with spherical geometry using back-to-back fisheye image pairs
- Un-OmniMVS network enhances inference speed through a novel feature extractor incorporating frequency attention and a variance-based light cost volume
- Experimental results show that the unsupervised solution performs comparably to state-of-the-art supervised methods and exhibits superior generalization capabilities with real-world data

## Summary

In the realm of computer vision, omnidirectional multi-view stereo (MVS) vision holds great appeal due to its ability to provide an ultra-wide field-of-view (FoV), allowing machines to perceive their 360-degree 3D surroundings. However, traditional solutions in this domain often necessitate costly dense depth labels for supervision, rendering them impractical for real-world applications. In response to this challenge, a groundbreaking approach is introduced in this paper: the first unsupervised omnidirectional MVS framework based on multiple fisheye images. The core concept of this innovative framework involves projecting all input images to a virtual view center and creating two panoramic images with spherical geometry by combining pairs of back-to-back fisheye images. Additionally, the authors propose Un-OmniMVS, an efficient unsupervised omnidirectional MVS network designed to enhance inference speed through two key components. Firstly, a novel feature extractor incorporating frequency attention is introduced to capture both non-local Fourier features and local spatial features simultaneously, thereby enhancing feature representation. Secondly, a variance-based light cost volume is implemented to reduce computational complexity. Experimental results demonstrate that the performance of this unsupervised solution rivals that of state-of-the-art supervised methods while exhibiting superior generalization capabilities when applied to real-world data. Authored by Zisong Chen, Chunyu Lin, Nie Lang, Kang Liao, and Yao Zhao, this research represents a significant advancement in the field of omnidirectional MVS vision by offering an efficient and effective unsupervised framework for depth inference across wide-ranging applications.