# AssistantX: An LLM-Powered Proactive Assistant in Collaborative Human-Populated Environment

**Authors:** Nan Sun, Bo Mao, Yongchang Li, Lumeng Ma, Di Guo, Huaping Liu  
**arXiv:** [2409.17655](https://arxiv.org/abs/2409.17655)  
**Date:** 2024-09-26  
**Summary page:** https://www.summarizepaper.com/en/arxiv-id/2409.17655v1/

## Key points

- Increasing demand for intelligent assistants in human-populated environments has led to significant research in autonomous robotic systems.
- Traditional service robots and virtual assistants have limitations in real-world task execution, especially when human collaboration is required.
- Recent advancements in Large Language Models (LLMs) have enhanced reasoning and interaction capabilities of these systems.
- AssistantX is an LLM-powered proactive assistant designed for autonomous operation in a physical office environment.
- AssistantX utilizes the PPDR4X multi-agent architecture to enhance inference capabilities and collaboration awareness.
- Contributions of this work include the development of AssistantX, design of the PPDR4X architecture, and demonstration of its effectiveness in managing complex real-world scenarios.

## Summary

In response to the increasing demand for intelligent assistants in human-populated environments, significant research has been conducted in autonomous robotic systems. Traditional service robots and virtual assistants have faced challenges in real-world task execution due to limited dynamic reasoning and interaction capabilities, especially when human collaboration is required. Recent advancements in Large Language Models (LLMs) have paved the way for enhancing these systems, enabling more sophisticated reasoning and natural interaction abilities. Introducing AssistantX, an LLM-powered proactive assistant designed to operate autonomously in a physical office environment. Unlike conventional service robots, AssistantX utilizes a novel multi-agent architecture called PPDR4X, which enhances inference capabilities and collaboration awareness. By bridging the gap between virtual operations and physical interactions, AssistantX demonstrates robust performance in managing complex real-world scenarios. The contributions of this work include: 1) Development of AssistantX - a robotic assistant that assists users in achieving goals both in virtual environments (e.g., engaging in conversations for assistance with tasks like printing or ordering takeout online) and physical environments (e.g., transferring paper files between individuals or picking up takeout). 2) Design of the PPDR4X multi-agent architecture that enables robots to reason logically and proficiently tackle problems similar to a human assistant. 3) Demonstration of the effectiveness of the proposed architecture for AssistantX - showcasing its ability to reactively respond to instructions, retrieve information from memory actively, and seek assistance proactively from team members within the office. The structure of this paper includes sections on related works, problem formulation, details of the proposed architecture for solving the problem at hand, comprehensive evaluation of the framework's effectiveness, and concluding remarks. Previous research on mobile robots operating in human-populated environments has focused on adaptability and human-robot interaction. Additionally, LLM-based multi-agent systems have shown significant progress with applications ranging from GUI operations for smart devices to communication between agents and humans. Overall, AssistantX represents a significant advancement in autonomous robotic systems by effectively combining virtual operations with physical interactions to handle complex office tasks efficiently. More information and videos can be accessed at https://assistantx-agent.github.io/AssistantX/.