Tree Search for Language Model Agents

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

1. Very close Another approach to the problem Not relevant

   Language Agent Tree Search Unifies Reasoning Acting and Planning in Language Models

   Andy Zhou, Kai Yan, Michal Shlapentokh-Rothman et al. · 2023 · Artificial Intelligence · AI summary available

   Language Agent Tree Search combines LLM planning, acting, and reasoning with environment feedback, like this paper’s tree search for interactive web agents.

   Abstract

   While large language models (LLMs) have demonstrated impressive performance on a range of decision-making tasks, they rely on simple acting processes and fall short of broad deployment as autonomous agents. We introduce LATS (Language Agent Tree Search), a general framework that synergizes the capabilities of LLMs in planning, acting, and reasoning. Drawing inspiration from Monte Carlo tree search in model-based reinforcement learning, LATS employs LLMs as agents, value functions, and optimizers, repurposing their latent strengths for enhanced decision-making. What is crucial in this method is the use of an environment for external feedback, which offers a more deliberate and adaptive proble…

2. Very close Another approach to the problem Not relevant

   Agent Q: Advanced Reasoning and Learning for Autonomous AI Agents

   Pranav Putta, Edmund Mills, Naman Garg et al. · 2024 · Artificial Intelligence · AI summary available

   Agent Q addresses multi-step reasoning in interactive web environments with a framework for agent learning and exploration, rather than inference-time tree search.

   Abstract

   Large Language Models (LLMs) have shown remarkable capabilities in natural language tasks requiring complex reasoning, yet their application in agentic, multi-step reasoning within interactive environments remains a difficult challenge. Traditional supervised pre-training on static datasets falls short in enabling autonomous agent capabilities needed to perform complex decision-making in dynamic settings like web navigation. Previous attempts to bridge this ga-through supervised fine-tuning on curated expert demonstrations-often suffer from compounding errors and limited exploration data, resulting in sub-optimal policy outcomes. To overcome these challenges, we propose a framework that comb…

3. Very close Broader context Not relevant

   SimuRA: Towards General Goal-Oriented Agent via Simulative Reasoning Architecture with LLM-Based World Model

   Mingkai Deng, Jinyu Hou, Yilin Shen et al. · 2025 · Artificial Intelligence · AI summary available

   SimuRA uses an LLM-based world model to simulate actions and plans for goal-oriented agents, a broader planning approach than web-environment tree search.

   Abstract

   AI agents built on large language models (LLMs) hold enormous promise, but current practice focuses on a one-task-one-agent approach, which not only falls short of scalability and generality, but also suffers from the fundamental limitations of autoregressive LLMs. On the other hand, humans are general agents who reason by mentally simulating the outcomes of their actions and plans. Moving towards a more general and powerful AI agent, we introduce SimuRA, a goal-oriented architecture for generalized agentic reasoning. Based on a principled formulation of optimal agent in any environment, \modelname overcomes the limitations of autoregressive reasoning by introducing a world model for plannin…

4. Close Same method, another problem Not relevant

   SELA: Tree-Search Enhanced LLM Agents for Automated Machine Learning

   Yizhou Chi, Yizhang Lin, Sirui Hong et al. · 2024 · Artificial Intelligence · AI summary available

   SELA applies Monte Carlo tree search to LLM agents, but uses it to optimize automated machine-learning pipelines rather than web tasks.

   Abstract

   Automated Machine Learning (AutoML) approaches encompass traditional methods that optimize fixed pipelines for model selection and ensembling, as well as newer LLM-based frameworks that autonomously build pipelines. While LLM-based agents have shown promise in automating machine learning tasks, they often generate low-diversity and suboptimal code, even after multiple iterations. To overcome these limitations, we introduce Tree-Search Enhanced LLM Agents (SELA), an innovative agent-based system that leverages Monte Carlo Tree Search (MCTS) to optimize the AutoML process. By representing pipeline configurations as trees, our framework enables agents to conduct experiments intelligently and it…

5. Close Same question Not relevant

   AgentOccam: A Simple Yet Strong Baseline for LLM-Based Web Agents

   Ke Yang, Yao Liu, Sapana Chaudhary et al. · 2024 · Artificial Intelligence · AI summary available

   AgentOccam studies LLM-based agents for automated web tasks, focusing on a simple baseline rather than inference-time tree search.

   Abstract

   Autonomy via agents using large language models (LLMs) for personalized, standardized tasks boosts human efficiency. Automating web tasks (like booking hotels within a budget) is increasingly sought after. Fulfilling practical needs, the web agent also serves as an important proof-of-concept example for various agent grounding scenarios, with its success promising advancements in many future applications. Prior research often handcrafts web agent strategies (e.g., prompting templates, multi-agent systems, search methods, etc.) and the corresponding in-context examples, which may not generalize well across all real-world scenarios. On the other hand, there has been limited study on the misali…

6. Close Review Not relevant

   A Survey on Large Language Model based Autonomous Agents

   Lei Wang, Chen Ma, Xueyang Feng et al. · 2023 · Artificial Intelligence · AI summary available

   A Survey on Large Language Model based Autonomous Agents reviews architectures and research on LLM-driven autonomous agents.

   Abstract

   Autonomous agents have long been a prominent research topic in the academic community. Previous research in this field often focuses on training agents with limited knowledge within isolated environments, which diverges significantly from the human learning processes, and thus makes the agents hard to achieve human-like decisions. Recently, through the acquisition of vast amounts of web knowledge, large language models (LLMs) have demonstrated remarkable potential in achieving human-level intelligence. This has sparked an upsurge in studies investigating autonomous agents based on LLMs. To harness the full potential of LLMs, researchers have devised diverse agent architectures tailored to di…

7. Close Broader context Not relevant

   Automating Thought of Search: A Journey Towards Soundness and Completeness

   Daniel Cao, Michael Katz, Harsha Kokel et al. · 2024 · Artificial Intelligence · AI summary available

   Automating Thought of Search studies LLM-guided planning and search, using generated code and human input to define a sound search space.

   Abstract

   Planning remains one of the last standing bastions for large language models (LLMs), which now turn their attention to search. Most of the literature uses the language models as world models to define the search space, forgoing soundness for the sake of flexibility. A recent work, Thought of Search (ToS), proposed defining the search space with code, having the language models produce that code. ToS requires a human in the loop, collaboratively producing a sound successor function and goal test. The result, however, is worth the effort: all the tested datasets were solved with 100% accuracy. At the same time LLMs have demonstrated significant progress in code generation and refinement for co…

8. Close Broader context Not relevant

   AutoAgents: A Framework for Automatic Agent Generation

   Guangyao Chen, Siwei Dong, Yu Shu et al. · 2023 · Artificial Intelligence · AI summary available

   AutoAgents generates and coordinates specialized agents for varied tasks, addressing adaptable task solving rather than web-agent tree search.

   Abstract

   Large language models (LLMs) have enabled remarkable advances in automated task-solving with multi-agent systems. However, most existing LLM-based multi-agent approaches rely on predefined agents to handle simple tasks, limiting the adaptability of multi-agent collaboration to different scenarios. Therefore, we introduce AutoAgents, an innovative framework that adaptively generates and coordinates multiple specialized agents to build an AI team according to different tasks. Specifically, AutoAgents couples the relationship between tasks and roles by dynamically generating multiple required agents based on task content and planning solutions for the current task based on the generated expert …