LabBook: Harnessing Experimental History for Efficient LLM-Driven Discovery

2026-09-30

AI Summary

This study addresses the limitations of evolutionary methods in LLM-driven scientific discovery, where context constraints risk overlooking historical evidence or full logging introduces redundancy. We propose a single-agent framework centered on a LabBook memory mechanism that decouples complete historical storage from selective context construction. By dynamically retrieving relevant evidence to generate novel solutions, the framework enables efficient exploration without explicit population-based or branching search structures. Integrating large language models, agent memory management, and retrieval-augmented generation, our approach significantly improves the quality-cost trade-off across 49 frontier computer science problems while maintaining competitiveness across diverse task categories. The source code is publicly available.

Abstract

Evolutionary approaches to LLM-driven discovery often generate new programs from a small set of selected ancestors. This keeps contexts manageable but can omit useful evidence from other experiments, whereas including the full experimental history produces long, redundant contexts. We introduce a simple, single-agent discovery harness built around LabBook, an agent-maintained memory that serves two complementary roles: guiding retrieval of relevant evidence from a complete experimental log and informing the generation of new solutions. At each iteration, the same agent combines its memory with retrieved evidence and jointly produces the next program and an updated LabBook. This separates complete history retention from selective context construction, without requiring an explicit population or branching search structure. On 49 Frontier-CS problems, LabBook improves the observed quality-cost trade-off over the evaluated evolutionary baselines with two backbones, while remaining competitive across nine additional mathematical, systems, and heuristic-design tasks. Code will be released at https://github.com/BoYuanVisionary/LabBook.

Problem

• LLM-driven discovery
• evolutionary approaches
• experimental history
• context management
• retrieval

Innovation

• LabBook
• LLM-driven discovery
• agent-maintained memory
• retrieval-augmented generation
• single-agent