Notes to Self: Can LLMs Benefit from Experiential Abstractions?

2026-07-22

AI Summary

This work investigates whether large language models can autonomously distill reusable abstract knowledge from problem-solving experiences to enhance reasoning capabilities, akin to human learners. The authors extract natural-language abstractions from the model’s solution trajectories on the MATH training set, construct a retrievable knowledge base, and integrate it during inference via retrieval-augmented reinforcement learning. This approach provides the first empirical validation that large language models can effectively self-extract and apply such abstractions, demonstrating strong transferability across both datasets and model architectures. The method yields significant performance gains on multiple mathematical and logical reasoning benchmarks, achieving results comparable to those obtained using abstractions distilled from a stronger teacher model.

Abstract

Humans distill experience into reusable abstractions, e.g., strategies and cautionary reminders, and apply them to gradually solve problems more effectively. We study whether Large Language Models (LLMs) can similarly benefit from such experiential abstractions. From LLMs' solution traces on the MATH training set, a stronger teacher or the LLMs themselves extract natural-language abstractions into a retrievable library. We explore two usage modes: (1) inference-time retrieval and (2) reinforcement learning (RL) with abstraction-augmented training prompts. Experiential abstractions improve LLM performance on mathematical and logical reasoning benchmarks. Self-extracted abstractions match teacher-extracted ones, and our abstraction usage framework can transfer to other datasets and models. These findings suggest LLMs can extract and apply experiential abstractions much as humans leverage distilled experience.

Problem

• experiential abstractions
• Large Language Models
• problem solving
• mathematical reasoning
• knowledge distillation

Innovation

• experiential abstractions
• large language models
• retrieval-augmented inference
• reinforcement learning
• knowledge distillation