# PADL: Language-Directed Physics-Based Character Control

**Authors:** Jordan Juravsky, Yunrong Guo, Sanja Fidler, Xue Bin Peng  
**arXiv:** [2301.13868](https://arxiv.org/abs/2301.13868)  
**Date:** 2023-01-31  
**Summary page:** https://www.summarizepaper.com/en/paper/2301.13868v1/

## Key points

- The development of systems that can synthesize natural and life-like motions for simulated characters has been a long-standing focus in computer animation.
- These systems must produce high-quality motions and provide an accessible and versatile interface through which users can direct a character's behaviors.
- Natural language provides a simple-to-use and expressive medium for specifying a user's intent, making it a promising tool for developing language-directed controllers for physics-based character animation.
- Jordan Juravsky, Yunrong Guo, Sanja Fidler, and Xue Bin Peng introduce PADL (Language-Directed Physics-Based Character Control) at the SIGGRAPH Asia 2022 Conference Papers.
- PADL leverages recent innovations in natural language processing (NLP) to develop a system for language-directed physics-based character animation.
- The central contribution of this work is a system that enables users to direct the behaviors of physically simulated characters using natural language commands.
- PADL trains control policies to map from high-level language commands to low-level motor commands that enable a character to reproduce corresponding skills.
- The system uses an adversarial imitation learning approach that allows policies to reproduce diverse arrays of skills while grounding each skill in language commands.
- Additionally, PADL presents a language-based multi-task aggregation model that selects between task-specific policies according to given commands, allowing users to easily direct characters to perform various high level tasks via natural language.
- The authors present one of the first systems capable of effectively leveraging language commands to direct full body physically simulated characters to perform complex motor skills.
- Their framework demonstrates effective use of NLP techniques in developing accessible interfaces for directing character behavior and offers promising potential for downstream applications in computer animation.
- Overall, PADL represents an important step towards developing more intuitive and versatile interfaces for controlling physically simulated characters using natural language commands.
- The code is available on GitHub at https://github.com/nvtlabs/PADL.

## Summary

The development of systems that can synthesize natural and life-like motions for simulated characters has been a long-standing focus in computer animation. To be useful for downstream applications, these systems must not only produce high-quality motions but also provide an accessible and versatile interface through which users can direct a character's behaviors. Natural language provides a simple-to-use and expressive medium for specifying a user's intent, making it a promising tool for developing language-directed controllers for physics-based character animation. In this work presented at the SIGGRAPH Asia 2022 Conference Papers, Jordan Juravsky, Yunrong Guo, Sanja Fidler, and Xue Bin Peng introduce PADL (Language-Directed Physics-Based Character Control), which leverages recent innovations in natural language processing (NLP) to develop a system for language-directed physics-based character animation. The central contribution of this work is a system that enables users to direct the behaviors of physically simulated characters using natural language commands. PADL trains control policies to map from high-level language commands to low-level motor commands that enable a character to reproduce corresponding skills. The system uses an adversarial imitation learning approach that allows policies to reproduce diverse arrays of skills while grounding each skill in language commands. Additionally, PADL presents a language-based multi-task aggregation model that selects between task-specific policies according to given commands, allowing users to easily direct characters to perform various high level tasks via natural language. The authors present one of the first systems capable of effectively leveraging language commands to direct full body physically simulated characters to perform complex motor skills. Their framework demonstrates effective use of NLP techniques in developing accessible interfaces for directing character behavior and offers promising potential for downstream applications in computer animation. Overall, PADL represents an important step towards developing more intuitive and versatile interfaces for controlling physically simulated characters using natural language commands. The code is available on GitHub at https://github.com/nvtlabs/PADL.