# Symphony: Learning Realistic and Diverse Agents for Autonomous Driving Simulation

**Authors:** Maximilian Igl, Daewoo Kim, Alex Kuefler, Paul Mougin, Punit Shah, Kyriacos Shiarlis, Dragomir Anguelov, Mark Palatucci, Brandyn White, Shimon Whiteson  
**arXiv:** [2205.03195](https://arxiv.org/abs/2205.03195)  
**Date:** 2022-05-06  
**Summary page:** https://www.summarizepaper.com/en/paper/2205.03195v1/

## Key points

- Simulation is crucial for expediting the development of autonomous driving technology
- Incorporating models of human road users interacting with autonomous vehicles is essential for realistic simulations
- Learning from demonstration (LfD) using observed trajectories can provide valuable training data
- Existing LfD methods often produce policies leading to collisions or veering off the road
- Symphony framework, developed by Maximilian Igl, Daewoo Kim, and team, enhances realism in simulations through a combination of conventional policies and parallel beam search
- Beam search dynamically refines policies by pruning branches receiving unfavorable evaluations from a discriminator
- Potential downside of reduced diversity addressed by Symphony's hierarchical approach separating agent behavior into goal generation and conditioning
- Experimental results on proprietary and open Waymo datasets validate Symphony's effectiveness in generating more realistic and diverse agent behaviors compared to baseline approaches

## Summary

In the field of autonomous driving, simulation plays a vital role in expediting the development process. To ensure that simulations are realistic, it is essential to incorporate models of human road users who interact with autonomous vehicles. One approach to obtaining these models is through learning from demonstration (LfD), where trajectories observed by vehicles on the road are used as training data. However, existing LfD methods often fall short, producing policies that result in frequent collisions or veering off the road. To address this challenge, a team of researchers led by Maximilian Igl, Daewoo Kim, and their colleagues have introduced Symphony—a novel framework designed to enhance realism in autonomous driving simulations. Symphony leverages a combination of conventional policies and a parallel beam search to significantly improve the fidelity of agent behavior. The beam search dynamically refines these policies by pruning branches that receive unfavorable evaluations from a discriminator. While the beam search enhances realism, there is a potential downside in terms of diversity—the ability of agents to cover a wide range of realistic behaviors. Pruning actions during training can lead to mode collapse, where agents converge on limited behaviors. To mitigate this issue, Symphony adopts a hierarchical approach that separates agent behavior into goal generation and goal conditioning. By incorporating goals into the learning process, Symphony ensures that agent diversity is preserved throughout adversarial training and is not compromised by the beam search. Experimental results conducted on both proprietary and open Waymo datasets validate the effectiveness of Symphony in generating more realistic and diverse agent behaviors compared to several baseline approaches. This research, accepted for presentation at ICRA-2022, represents a significant advancement in enhancing the capabilities of autonomous driving simulations through innovative machine learning techniques.