# Photo-SLAM: Real-time Simultaneous Localization and Photorealistic Mapping for Monocular, Stereo, and RGB-D Cameras

**Authors:** Huajian Huang, Longwei Li, Hui Cheng, Sai-Kit Yeung  
**arXiv:** [2311.16728](https://arxiv.org/abs/2311.16728)  
**Date:** 2023-11-28  
**Summary page:** https://www.summarizepaper.com/en/paper/2311.16728v1/

## Key points

- Photo-SLAM is a cutting-edge SLAM framework developed by Huajian Huang, Longwei Li, Hui Cheng, and Sai-Kit Yeung
- Integration of neural rendering with the SLAM system has yielded promising results in achieving joint localization and photorealistic view reconstruction
- Introduction of hyper primitives map leveraging explicit geometric features for precise localization and implicit photometric features for texture information capture
- Actively densifying hyper primitives based on geometric features and implementing Gaussian-Pyramid-based training method for enhanced photorealistic mapping performance
- Extensive experiments demonstrate superior capabilities of Photo-SLAM compared to current state-of-the-art SLAM systems, achieving 30% higher Peak Signal-to-Noise Ratio (PSNR) and significantly faster rendering speed on Replica dataset
- Real-time operational efficiency showcased when deployed on embedded platforms like Jetson AGX Orin, highlighting potential for diverse robotics applications
- Represents a significant advancement in simultaneous localization and mapping field by offering unparalleled performance in online photorealistic mapping across different camera types

## Summary

Photo-SLAM is a cutting-edge SLAM framework developed by Huajian Huang, Longwei Li, Hui Cheng, and Sai-Kit Yeung that revolutionizes simultaneous localization and photorealistic mapping for monocular, stereo, and RGB-D cameras. The integration of neural rendering with the SLAM system has yielded promising results in achieving joint localization and photorealistic view reconstruction. This innovative framework introduces a hyper primitives map that leverages both explicit geometric features for precise localization and implicit photometric features to capture the texture information of the observed environment. By actively densifying hyper primitives based on geometric features and implementing a Gaussian-Pyramid-based training method to progressively learn multi-level features, Photo-SLAM significantly enhances its photorealistic mapping performance. Extensive experiments conducted with various datasets including monocular, stereo, and RGB-D have demonstrated the superior capabilities of Photo-SLAM compared to current state-of-the-art SLAM systems. For instance, the Peak Signal-to-Noise Ratio (PSNR) achieved by Photo-SLAM is 30% higher than existing systems, while its rendering speed surpasses them by hundreds of times when tested on the Replica dataset. Furthermore, Photo-SLAM showcases real-time operational efficiency when deployed on embedded platforms like Jetson AGX Orin, showcasing its potential for diverse robotics applications. In conclusion, Photo-SLAM represents a significant advancement in the field of simultaneous localization and mapping by offering unparalleled performance in online photorealistic mapping across different camera types. Its innovative approach combining explicit geometric features with implicit photometric representations sets a new standard for real-time visual reconstruction in robotics applications.