# Configurable 3D Scene Synthesis and 2D Image Rendering with Per-Pixel Ground Truth using Stochastic Grammars

**Authors:** Chenfanfu Jiang, Siyuan Qi, Yixin Zhu, Siyuan Huang, Jenny Lin, Lap-Fai Yu, Demetri Terzopoulos, Song-Chun Zhu  
**arXiv:** [1704.00112](https://arxiv.org/abs/1704.00112)  
**Date:** 2017-04-01  
**Summary page:** https://www.summarizepaper.com/en/paper/1704.00112v3/

## Key points

- Authors propose a systematic learning-based approach for generating synthetic 3D scenes and photorealistic 2D images
- Pipeline of algorithms can automatically generate diverse indoor scenes using a stochastic grammar and physics-based rendering
- Precise customization and control of scene attributes is possible
- Renders realistic RGB images while synthesizing detailed per-pixel ground truth data such as depth, surface normal, object identity, material information, and environmental factors
- Synthesized dataset improves performance in machine learning based scene understanding tasks
- Provides benchmarks for trained models through controllable modifications of object attributes and scene properties
- Paper accepted in the International Journal of Computer Vision (IJCV) in 2018

## Summary

The authors propose a systematic learning-based approach for generating large quantities of synthetic 3D scenes and photorealistic 2D images, along with ground truth information, to train and evaluate computer vision and robotics algorithms. They develop a pipeline of algorithms that can automatically generate diverse indoor scenes using a stochastic grammar and physics-based rendering. The pipeline allows for precise customization and control of scene attributes. It renders realistic RGB images while synthesizing detailed per-pixel ground truth data such as depth, surface normal, object identity, material information and environmental factors. The authors demonstrate the value of their synthesized dataset by improving performance in various machine learning based scene understanding tasks and providing benchmarks for trained models through controllable modifications of object attributes and scene properties. The paper has been accepted in the International Journal of Computer Vision (IJCV) in 2018.