# Scaffold-GS: Structured 3D Gaussians for View-Adaptive Rendering

**Authors:** Tao Lu, Mulin Yu, Linning Xu, Yuanbo Xiangli, Limin Wang, Dahua Lin, Bo Dai  
**arXiv:** [2312.00109](https://arxiv.org/abs/2312.00109)  
**Date:** 2023-11-30  
**Summary page:** https://www.summarizepaper.com/en/paper/2312.00109v1/

## Key points

- Neural rendering methods have advanced in achieving photo-realistic 3D scene rendering
- The 3D Gaussian Splatting technique combines primitive-based and volumetric representations for high-quality rendering with improved speed
- However, it generates redundant Gaussians that do not consider the underlying scene geometry, leading to a less robust model
- Scaffold-GS is a new method that uses anchor points to distribute local 3D Gaussians throughout the scene and predicts their attributes based on viewing direction and distance
- It incorporates anchor growing and pruning strategies to prioritize important neural Gaussians for improved scene coverage
- Experimental results show that Scaffold-GS reduces redundant Gaussians while maintaining high quality renderings at an efficient speed
- It can accommodate scenes with varying levels of detail and view dependent observations without compromising rendering speed.

## Summary

Neural rendering methods have made significant advancements in achieving photo-realistic 3D scene rendering in various academic and industrial applications. One such method, the 3D Gaussian Splatting technique, combines the advantages of both primitive-based representations and volumetric representations to achieve high-quality rendering with improved speed. However, a drawback of this method is its tendency to generate heavily redundant Gaussians that attempt to fit every training view without considering the underlying scene geometry. This leads to a less robust model that is sensitive to changes in viewpoint, texture-less areas, and lighting effects. To address this limitation, a new method called Scaffold-GS has been introduced. Scaffold-GS utilizes anchor points to distribute local 3D Gaussians throughout the scene. These Gaussians' attributes are predicted on-the-fly based on the viewing direction and distance within the view frustum. The method also incorporates anchor growing and pruning strategies that prioritize important neural Gaussians for improved scene coverage reliably. Experimental results demonstrate that Scaffold-GS effectively reduces redundant Gaussians while maintaining high quality renderings at an efficient speed. Additionally, it showcases an enhanced capability to accommodate scenes with varying levels of detail and view dependent observations without compromising rendering speed. In conclusion, Scaffold-GS presents a structured approach using 3D Gaussians for view adaptive rendering which addresses the limitations of previous techniques by incorporating anchor points and dynamic attribute prediction for improved scene coverage while delivering high quality renderings at an efficient speed.