# MotionGPT: Finetuned LLMs are General-Purpose Motion Generators

**Authors:** Yaqi Zhang, Di Huang, Bin Liu, Shixiang Tang, Yan Lu, Lu Chen, Lei Bai, Qi Chu, Nenghai Yu, Wanli Ouyang  
**arXiv:** [2306.10900](https://arxiv.org/abs/2306.10900)  
**Date:** 2023-06-19  
**Summary page:** https://www.summarizepaper.com/en/paper/2306.10900v1/

## Key points

- Generating realistic human motion from given action descriptions is important in the digital human industry
- Recent advancements have allowed for generating motion directly from textual action descriptions
- These methods often only support a single modality of the control signal, limiting their application in real-world scenarios
- MotionGPT is introduced as a Motion General-Purpose generaTor that can utilize multimodal control signals to generate consecutive human motions
- Multimodal control signals include text and single-frame poses
- MotionGPT treats multimodal signals as special input tokens in large language models (LLMs)
- Multimodal control signals are quantized into discrete codes and formulated into a unified prompt instruction for LLMs to generate desired motion
- MotionGPT can generate human motion with multimodal control signals by tuning just 0.4% of the LLM parameters
- This is the first method to generate human motion using multimodal control signals, opening up new possibilities and directions for research in this field
- The authors plan to release the codes upon acceptance of their work

## Summary

Generating realistic human motion from given action descriptions has become increasingly important in the digital human industry. Recent advancements have allowed for impressive results in generating motion directly from textual action descriptions. However, these methods often only support a single modality of the control signal, limiting their application in real-world scenarios. To address this limitation, this paper introduces MotionGPT, a Motion General-Purpose generaTor that can utilize multimodal control signals to generate consecutive human motions. These multimodal control signals include text and single-frame poses. MotionGPT achieves this by treating the multimodal signals as special input tokens in large language models (LLMs). The process begins by quantizing the multimodal control signals into discrete codes. These codes are then formulated into a unified prompt instruction, which is used to instruct the LLMs to generate the desired motion. Remarkably, MotionGPT demonstrates its ability to generate human motion with multimodal control signals by tuning just 0.4% of the LLM parameters. This approach is significant as it is the first method to generate human motion using multimodal control signals. It opens up new possibilities and directions for research in this field. The authors hope that MotionGPT will shed light on this new direction and contribute to further advancements in generating realistic human motion. The authors plan to release the codes upon acceptance of their work. Overall, this paper presents a comprehensive and innovative approach to generating human motion using multimodal control signals, showcasing its potential impact on the digital human industry.