# Super-NeRF: View-consistent Detail Generation for NeRF super-resolution

**Authors:** Yuqi Han, Tao Yu, Xiaohang Yu, Yuwang Wang, Qionghai Dai  
**arXiv:** [2304.13518](https://arxiv.org/abs/2304.13518)  
**Date:** 2023-04-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2304.13518v1/

## Key points

- Neural radiance field (NeRF) is a powerful method for modeling 3D scenes and generating novel views
- Existing NeRF approaches focus on image resolution without considering fine details under limited input resolution
- Super-NeRF is a proposed NeRF super-resolution method that generates high-resolution NeRF representations from low-resolution inputs
- Super-NeRF utilizes multi-view low-resolution images and a consistency-controlling super-resolution module to generate view-consistent high-resolution details
- Latent codes control the convergence of 2D super-resolved images towards the desired output, optimized synergistically with the target Super NeRF representation
- Super-NeRF achieves state-of-the-art performance in NeRF super-resolution by ensuring cross-view consistency and generating high-resolution details
- Mutual learning policy maintains cross-view consistency between the consistency controlling super-resolution module (CCSR) and NeRF
- Consistency enforcing module (CEM) ensures that 2D super-resolved results conform to low-resolution images
- Super-NeRF is effective in various radiance field construction scenarios, including planar light fields, spherical light fields, 3D human faces, and AI-generated light fields
- Super-NeRF demonstrates robustness and adaptive adjustment based on camera settings

## Summary

In recent years, the neural radiance field (NeRF) has emerged as a powerful method for modeling 3D scenes and generating high-quality novel views. However, existing NeRF-based approaches primarily focus on utilizing image resolution to generate new views, without considering the generation of fine details under limited input resolution. To address this limitation, we propose a NeRF super-resolution method called Super-NeRF. Super-NeRF aims to generate high-resolution NeRF representations from low-resolution inputs. By leveraging the concept of image super-resolution, Super-NeRF offers an effective solution for generating high-resolution implicit representations of 3D scenes. This approach holds great potential for various applications but has been largely unexplored thus far. Our method utilizes multi-view low-resolution images as input and constructs a consistency-controlling super-resolution module within Super-NeRF. This module generates view-consistent high-resolution details for NeRF by introducing an optimizable latent code for each low-resolution input image. These latent codes control the convergence of 2D super resolution images towards the desired view consistent output. Additionally, the latent codes are optimized synergistically with the target Super NeRF representation to fully utilize the inherent view consistency constraint in NeRF construction. We validate the effectiveness of Super NeRF on synthetic, real world and AI generated NeRF datasets. Our method achieves state of the art performance in Ne RF super resolution by generating high resolution details and ensuring cross view consistency. To achieve these results, we propose a mutual learning policy that maintains cross view consistency between our consistency controlling super resolution module (CCSR) and Ne RF . We also introduce a consistency enforcing module (CEM) to ensure that the 2D super resolved results conform to the low resolution images. Two individual MLPs are designed: LR Ne RF serves as coarse guidance for view consistency in CCSR , while HR Ne RF is optimized concurrently with CCSR to learn view consistent detail information . A trade off factor is introduced to conduct weighted loss for LR Ne RF and HR Ne RF , gradually decreasing the effect of LR Ne RF on image generation as the iteration progresses . In our experiments , we demonstrate the effectiveness of Super -Ne RF in various radiance field construction scenarios , including planar light fields , 360 degree synthesized spherical light fields , 3D human faces , and AI generated light fields . We also test scalability of Super -Ne RF in different camera settings , including input images with all low resolution or a mix of low and high resolution . Our method proves robust to various scenes and exhibits adaptive adjustment based on camera settings . In summary , our contributions include proposal of Super -Ne RF as generative Nerf super -resolution method for view consistent and high resolution novel view synthesis using only low resolution images as input . We introduce a consistency controlling SR module that successfully explores solution space for generating details while maintaining multi -view consistency .Super -Nerf demonstrates strong generalization capacity across different types of light fields and supports hybrid input resolution settings based on practical requirements . The paper provides further details on related work in Section 2 , presents framework of Super -Nerf and details its consistency controlling SR module in Section 3 . Experimental results are presented in Section 4 to validate effectiveness of Super -Nerf across various scenarios . Finally , Section 5 summarizes paper's findings .