A Benchmarking Framework for Interactive 3D Applications in the Cloud

Abstract: With the growing popularity of cloud gaming and cloud virtual reality (VR), interactive 3D applications have become a major type of workloads for the cloud. However, despite their growing importance, there is limited public research on how to design cloud systems to efficiently support these applications, due to the lack of an open and reliable research infrastructure, including benchmarks and performance analysis tools. The challenges of generating human-like inputs under various system/application randomness and dissecting the performance of complex graphics systems make it very difficult to design such an infrastructure. In this paper, we present the design of a novel cloud graphics rendering research infrastructure, Pictor. Pictor employs AI to mimic human interactions with complex 3D applications. It can also provide in-depth performance measurements for the complex software and hardware stack used for cloud 3D graphics rendering. With Pictor, we designed a benchmark suite with six interactive 3D applications. Performance analyses were conducted with these benchmarks to characterize 3D applications in the cloud and reveal new performance bottlenecks. To demonstrate the effectiveness of Pictor, we also implemented two optimizations to address two performance bottlenecks discovered in a state-of-the-art cloud 3D-graphics rendering system, which improved the frame rate by 57.7% on average.