# Memento: Fine-tuning LLM Agents without Fine-tuning LLMs

**Authors:** Huichi Zhou, Yihang Chen, Siyuan Guo, Xue Yan, Kin Hei Lee, Zihan Wang, Ka Yiu Lee, Guchun Zhang, Kun Shao, Linyi Yang, Jun Wang  
**arXiv:** [2508.16153](https://arxiv.org/abs/2508.16153)  
**Date:** 2025-08-22  
**Summary page:** https://www.summarizepaper.com/en/paper/2508.16153v2/

## Key points

- Introduction of a novel learning paradigm for Adaptive Large Language Model (LLM) agents
- Eliminates the need for fine-tuning underlying LLMs
- Enables low-cost continual adaptation through memory-based online reinforcement learning
- Formalized as a Memory-augmented Markov Decision Process (M-MDP) with neural case-selection policy
- Incorporation of external tools to overcome context limitations and computational bottlenecks
- Agentic Reinforcement Learning (Agentic RL) for dynamic agent-environment reasoning
- Experimental results demonstrate effectiveness of proposed Memento agent model on datasets like Deep Researcher and GAIA
- Outperforms existing frameworks in long-horizon planning and tool orchestration tasks
- Capabilities showcased in real-time web research, evidence retrieval, cross-page synthesis, and multi-hop reasoning tasks
- Strong performance on Humanity's Last Exam (HLE) for complex reasoning tasks within specialized domains
- Incorporation of case-based reasoning into planning processes enhances overall performance and offers scalable pathway for developing generalist LLM agents

## Summary

In this paper, we introduce a novel learning paradigm for Adaptive Large Language Model (LLM) agents that eliminates the need for fine-tuning the underlying LLMs. Existing approaches often rely on static workflows or computationally intensive gradient updates of model parameters. Our proposed method enables low-cost continual adaptation through memory-based online reinforcement learning, formalized as a Memory-augmented Markov Decision Process (M-MDP) with a neural case-selection policy. Past experiences are stored in an episodic memory, and the policy is continually updated based on environmental feedback. The tool-augmented LLM approach incorporates external tools to overcome context limitations and computational bottlenecks. Recent works have proposed multi-agent pipelines to coordinate specialized agents via dialogue for long-horizon tasks. Agentic Reinforcement Learning (Agentic RL) has emerged as a promising training paradigm, shifting LLM training towards dynamic agent-environment reasoning. Experimental results on datasets like Deep Researcher and GAIA demonstrate the effectiveness of the proposed Memento agent model. Memento achieves top rankings on validation sets and outperforms existing frameworks in long-horizon planning and tool orchestration tasks. Real-time web research, evidence retrieval, cross-page synthesis, and multi-hop reasoning tasks showcase the capabilities of Memento augmented with MCP tools. Furthermore, performance evaluations on Humanity's Last Exam (HLE) highlight Memento's ability to excel in complex reasoning tasks within specialized domains. By incorporating case-based reasoning into planning processes, Memento demonstrates strong overall performance and offers a scalable pathway for developing generalist LLM agents capable of continuous learning without gradient updates. Overall, our proposed approach presents a scalable and efficient solution for developing adaptive LLM agents capable of real-time learning without the need for fine-tuning LLMs. The incorporation of external tools and case-based reasoning enhances decision-making processes and performance across various challenging tasks, advancing machine learning towards open-ended skill acquisition scenarios.