Learning to Reason and Memorize with Self-Notes

Ranked by how close their title and abstract are to this paper’s. Select some to read them together.

Closest papers

1. Close Not relevant

   Chain-of-Thought Reasoning is a Policy Improvement Operator

   Hugh Zhang, David C. Parkes · 2023 · Machine Learning · AI summary available

   Abstract

   Large language models have astounded the world with fascinating new capabilities. However, they currently lack the ability to teach themselves new skills, relying instead on being trained on large amounts of human-generated data. We introduce SECToR (Self-Education via Chain-of-Thought Reasoning), a proof-of-concept demonstration that language models can successfully teach themselves new skills using chain-of-thought reasoning. Inspired by previous work in both reinforcement learning (Silver et al., 2017) and human cognition (Kahneman, 2011), SECToR first uses chain-of-thought reasoning to slowly think its way through problems. SECToR then fine-tunes the model to generate those same answers,…

2. Close Not relevant

   STaR: Bootstrapping Reasoning With Reasoning

   Eric Zelikman, Yuhuai Wu, Jesse Mu et al. · 2022 · Machine Learning · AI summary available

   Abstract

   Generating step-by-step "chain-of-thought" rationales improves language model performance on complex reasoning tasks like mathematics or commonsense question-answering. However, inducing language model rationale generation currently requires either constructing massive rationale datasets or sacrificing accuracy by using only few-shot inference. We propose a technique to iteratively leverage a small number of rationale examples and a large dataset without rationales, to bootstrap the ability to perform successively more complex reasoning. This technique, the "Self-Taught Reasoner" (STaR), relies on a simple loop: generate rationales to answer many questions, prompted with a few rationale exam…

3. Close Not relevant

   Scaling Up RL: Unlocking Diverse Reasoning in LLMs via Prolonged Training

   Mingjie Liu, Shizhe Diao, Jian Hu et al. · 2025 · Machine Learning · AI summary available

   Abstract

   Recent advancements in reasoning-focused language models such as OpenAI's O1 and DeepSeek-R1 have shown that scaling test-time computation-through chain-of-thought reasoning and iterative exploration-can yield substantial improvements on complex tasks like mathematics and code generation. These breakthroughs have been driven by large-scale reinforcement learning (RL), particularly when combined with verifiable reward signals that provide objective and grounded supervision. In this report, we investigate the effects of prolonged reinforcement learning on a small language model across a diverse set of reasoning domains. Our work identifies several key ingredients for effective training, includ…

4. Close Not relevant

   Alice in Wonderland: Simple Tasks Showing Complete Reasoning Breakdown in State-Of-the-Art Large Language Models

   Marianna Nezhurina, Lucia Cipolina-Kun, Mehdi Cherti et al. · 2024 · Machine Learning · AI summary available

   Abstract

   Large Language Models (LLMs) are often described as being instances of foundation models - that is, models that transfer strongly across various tasks and conditions in few-show or zero-shot manner, while exhibiting scaling laws that predict function improvement when increasing the pre-training scale. These claims of excelling in different functions and tasks rely on measurements taken across various sets of standardized benchmarks showing high scores for such models. We demonstrate here a dramatic breakdown of function and reasoning capabilities of state-of-the-art models trained at the largest available scales which claim strong function, using a simple, short, conventional common sense pr…

5. Close Not relevant

   Scaling up Test-Time Compute with Latent Reasoning: A Recurrent Depth Approach

   Jonas Geiping, Sean McLeish, Neel Jain et al. · 2025 · Machine Learning · AI summary available

   Abstract

   We study a novel language model architecture that is capable of scaling test-time computation by implicitly reasoning in latent space. Our model works by iterating a recurrent block, thereby unrolling to arbitrary depth at test-time. This stands in contrast to mainstream reasoning models that scale up compute by producing more tokens. Unlike approaches based on chain-of-thought, our approach does not require any specialized training data, can work with small context windows, and can capture types of reasoning that are not easily represented in words. We scale a proof-of-concept model to 3.5 billion parameters and 800 billion tokens. We show that the resulting model can improve its performanc…

6. Close Not relevant

   Quantifying Memorization Across Neural Language Models

   Nicholas Carlini, Daphne Ippolito, Matthew Jagielski et al. · 2022 · Machine Learning · AI summary available

   Abstract

   Large language models (LMs) have been shown to memorize parts of their training data, and when prompted appropriately, they will emit the memorized training data verbatim. This is undesirable because memorization violates privacy (exposing user data), degrades utility (repeated easy-to-memorize text is often low quality), and hurts fairness (some texts are memorized over others). We describe three log-linear relationships that quantify the degree to which LMs emit memorized training data. Memorization significantly grows as we increase (1) the capacity of a model, (2) the number of times an example has been duplicated, and (3) the number of tokens of context used to prompt the model. Surpris…

7. Close Not relevant

   Show Your Work: Scratchpads for Intermediate Computation with Language Models

   Maxwell Nye, Anders Johan Andreassen, Guy Gur-Ari et al. · 2021 · Machine Learning · AI summary available

   Abstract

   Large pre-trained language models perform remarkably well on tasks that can be done "in one pass", such as generating realistic text or synthesizing computer programs. However, they struggle with tasks that require unbounded multi-step computation, such as adding integers or executing programs. Surprisingly, we find that these same models are able to perform complex multi-step computations -- even in the few-shot regime -- when asked to perform the operation "step by step", showing the results of intermediate computations. In particular, we train transformers to perform multi-step computations by asking them to emit intermediate computation steps into a "scratchpad". On a series of increasin…

8. Close Not relevant

   Titans: Learning to Memorize at Test Time

   Ali Behrouz, Peilin Zhong, Vahab Mirrokni · 2024 · Machine Learning · AI summary available

   Abstract

   Over more than a decade there has been an extensive research effort on how to effectively utilize recurrent models and attention. While recurrent models aim to compress the data into a fixed-size memory (called hidden state), attention allows attending to the entire context window, capturing the direct dependencies of all tokens. This more accurate modeling of dependencies, however, comes with a quadratic cost, limiting the model to a fixed-length context. We present a new neural long-term memory module that learns to memorize historical context and helps attention to attend to the current context while utilizing long past information. We show that this neural memory has the advantage of fas…