# Wonderful Team: Zero-Shot Physical Task Planning with Visual LLMs

**Authors:** Zidan Wang, Rui Shen, Bradly Stadie  
**arXiv:** [2407.19094](https://arxiv.org/abs/2407.19094)  
**Date:** 2024-07-26  
**Summary page:** https://www.summarizepaper.com/en/paper/2407.19094v6/

## Key points

- Paper presents a state-of-the-art multi-agent Vision Large Language Model (VLLM) framework for high-level robotic planning in a zero-shot regime
- Utilizes an image of the robot's surroundings and a task description to generate action sequences for completing novel tasks
- Integrates VLLMs throughout the entire planning process, outperforming traditional methods that rely on separate vision systems
- Demonstrates significant improvements over previous approaches such as NLaP and Trajectory Generators
- Highlights the potential of VLLMs in enhancing robotic planning efficiency and effectiveness, calling for further exploration by researchers and practitioners

## Summary

The paper "Wonderful Team: Zero-Shot Physical Task Planning with Visual LLMs" by Zidan Wang, Rui Shen, and Bradly Stadie presents a state-of-the-art multi-agent Vision Large Language Model (VLLM) framework for high-level robotic planning in a zero-shot regime. This cutting-edge approach utilizes an image of the robot's surroundings and a task description to generate action sequences for completing novel tasks. Unlike traditional methods that rely on separate vision systems, Wonderful Team integrates VLLMs throughout the entire planning process to achieve superior performance in real-world semantic and physical planning tasks. The results demonstrate significant improvements over previous approaches such as NLaP and Trajectory Generators, highlighting the potential of VLLMs in enhancing robotic planning efficiency and effectiveness. This work serves as a compelling call to action for researchers and practitioners to explore the capabilities of VLLMs in addressing high-level robotic planning problems.