# AutoTAMP: Autoregressive Task and Motion Planning with LLMs as Translators and Checkers

**Authors:** Yongchao Chen, Jacob Arkin, Yang Zhang, Nicholas Roy, Chuchu Fan  
**arXiv:** [2306.06531](https://arxiv.org/abs/2306.06531)  
**Date:** 2023-06-10  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2306.06531v1/

## Key points

- Robots' ability to comprehend, strategize, and carry out intricate, long-term tasks articulated in natural language is crucial for effective human-robot interaction.
- Recent advancements in large language models (LLMs) show promise in translating natural language into sequences of actions for robots to execute complex tasks.
- A new approach involves few-shot translation from natural language task descriptions to an intermediary task representation, which can be utilized by a traditional task-and-motion planning (TAMP) algorithm to collaboratively solve both the task and motion plan.
- Automatic detection and correction of syntactic and semantic errors through autoregressive re-prompting enhance the translation process and result in notable enhancements in task completion rates.
- The newly proposed method showcased significant superiority over existing methods that employ LLMs as planners in navigating complex task domains.
- Challenges persist when dealing with complexities such as temporally-dependent multi-step actions, action sequence optimization, and task constraints despite efforts made towards enhancing executability through feedback mechanisms and verifying sub-task sequences' executability within the framework.

## Summary

In the realm of effective human-robot interaction, the ability of robots to comprehend, strategize, and carry out intricate, long-term tasks articulated in natural language is crucial. Recent advancements in large language models (LLMs) have shown promise in translating natural language into sequences of actions for robots to execute complex tasks. However, existing approaches often either directly translate natural language into robot trajectories or break down the inference process by segmenting language into task sub-goals and relying on a motion planner to execute each sub-goal. When faced with complex environmental and temporal constraints, the joint performance of inference over planning tasks alongside motion plans using traditional task-and-motion planning (TAMP) algorithms becomes necessary, rendering such factorization impractical. Rather than utilizing LLMs to directly plan task sub-goals, a new approach has emerged that involves few-shot translation from natural language task descriptions to an intermediary task representation. This intermediary representation can then be utilized by a TAMP algorithm to collaboratively solve both the task and motion plan. To enhance the translation process, automatic detection and correction of syntactic and semantic errors are implemented through autoregressive re-prompting, resulting in notable enhancements in task completion rates. The newly proposed method showcased significant superiority over several existing methods that employ LLMs as planners in navigating complex task domains. Additionally, efforts have been made towards addressing issues related to feedback mechanisms and verifying the executability of sub-task sequences within this framework. Despite previous research focusing on enhancing executability through connecting sub-tasks to control policy affordance functions or providing environmental feedback on robot actions, challenges persist when dealing with various complexities such as temporally-dependent multi-step actions, action sequence optimization, and task constraints. Furthermore, existing frameworks tend to segregate the planning problem by inferring a task plan separately from the motion plan using LLMs. This separation poses limitations when handling intricate tasks that require seamless integration between planning and execution processes. The continuous evolution and refinement of methodologies like AutoTAMP hold promise for advancing human-robot interaction capabilities by enabling robots to effectively interpret natural language instructions for executing complex tasks with precision and efficiency.