# MemGPT: Towards LLMs as Operating Systems

**Authors:** Charles Packer, Vivian Fang, Shishir G. Patil, Kevin Lin, Sarah Wooders, Joseph E. Gonzalez  
**arXiv:** [2310.08560](https://arxiv.org/abs/2310.08560)  
**Date:** 2023-10-12  
**Summary page:** https://www.summarizepaper.com/en/paper/2310.08560v1/

## Key points

- Large language models (LLMs) have advanced AI capabilities but are limited by their context windows
- MemGPT (Memory-GPT) proposes virtual context management inspired by hierarchical memory systems
- MemGPT intelligently manages different memory tiers to provide extended context within the LLM's window
- MemGPT utilizes interrupts to manage control flow between itself and the user
- MemGPT is evaluated in document analysis and multi-session chat domains
- In document analysis, MemGPT can analyze large documents exceeding the LLM's context window
- In multi-session chat, MemGPT creates conversational agents that remember and evolve dynamically through long-term interactions with users
- Code and data for experiments are available at https://memgpt.ai
- Previous studies have explored methods to improve context length in LLMs for coherent dialogues and question answering tasks
- Recursive summarization can generate concise representations over sliding windows but may lose relevant details or nuances
- Other approaches focus on improving LLMs' ability to attend to longer sequences using search and retrieval mechanisms like RAG systems with external databases or conversation logs for contextual relevance responses
- Various retrieval mechanisms can be integrated into MemGPT as part of its disk memory
- Engaging conversation openers that draw from user persona information are important for MemGPT, without working context, opener quality degrades significantly
- Dialogue stored only in recall memory does not affect opener generation since MemGPT generally does not search conversation history before generating an opener
- Current transformer models face challenges due to limited context windows in document analysis
- MemGPT addresses this limitation by effectively analyzing large documents that exceed the token limit set by GPT models
- Overall, MemGPT demonstrates improved performance compared to existing methods like recursive summarization or search/retrieval mechanisms employed by RAG systems

## Summary

Large language models (LLMs) have greatly advanced AI capabilities but are limited by their context windows, which hinders their performance in tasks like extended conversations and document analysis. To address this limitation, the authors propose virtual context management inspired by hierarchical memory systems in traditional operating systems. They introduce MemGPT (Memory-GPT), a system that intelligently manages different memory tiers to provide extended context within the LLM's limited window. MemGPT utilizes interrupts to manage control flow between itself and the user. The authors evaluate MemGPT in two domains: document analysis and multi-session chat. In document analysis, MemGPT can analyze large documents exceeding the underlying LLM's context window. In multi-session chat, MemGPT creates conversational agents that remember and evolve dynamically through long-term interactions with users. The authors provide the code and data for their experiments at https://memgpt.ai. In related work, previous studies have explored methods to improve context length in LLMs for coherent dialogues and question answering tasks. Recursive summarization has been used to generate concise representations over sliding windows, but it can result in loss of relevant details or nuances. Other approaches have focused on improving LLMs' ability to attend to longer sequences. Search and retrieval mechanisms, particularly in retrieval-augmented generation (RAG), have been incorporated into conversational agents using external databases or conversation logs for contextually relevant responses. Various retrieval mechanisms can be integrated into MemGPT as part of its disk memory. The authors also highlight the importance of engaging conversation openers that draw from user persona information. Without working context, MemGPT's openers significantly degrade in quality, while having dialogue stored only in recall memory does not affect the opener generation since MemGPT generally does not search conversation history before generating an opener. In terms of document analysis, current transformer models face challenges due to limited context windows. OpenAI's GPT models, for example, have a token limit of 32k; however, MemGPT addresses this limitation by effectively analyzing large documents that exceed the context window size limit set by these models. Overall, the proposed MemGPT system and its virtual context management technique demonstrate improved performance in tasks requiring extended contexts such as document analysis and multi-session chat conversations compared to existing methods such as recursive summarization or search/retrieval mechanisms employed by RAG systems with external databases or conversation logs for contextual relevance responses..