Deliberate Evolution: Agentic Reasoning for Sample-Efficient Symbolic Regression with LLMs

2026-06-02

AI Summary

Current large language model (LLM)-driven symbolic regression methods suffer from low sample efficiency due to their reliance solely on scalar feedback signals such as mean squared error (MSE). To address this limitation, this work proposes the Deliberate Evolution framework, which introduces an agent-based reasoning mechanism into symbolic regression for the first time. The framework decouples symbolic expression generation from search control by explicitly separating search direction guidance, error diagnosis, and experience reuse through adaptive operators, structural diagnostic tools, and trajectory-level reflective memory. Evaluated on the LLM-SRBench benchmark, the proposed method consistently outperforms existing LLM-based symbolic regression approaches while using only 40% of the standard sample budget, achieving superior performance across multiple scientific domains.

Abstract

Symbolic regression (SR) discovers compact mathematical expressions from data, yet recent LLM-based evolutionary methods remain sample-inefficient because they rely mainly on scalar feedback such as MSE. We identify a core limitation: existing methods conflate candidate proposal with search guidance, requiring the LLM to infer how to evolve an expression, diagnose its errors, and reuse past experience from a single score. To address this, we propose Deliberate Evolution (DE), an agentic framework that decouples symbolic generation from search control. DE guides LLM proposals with adaptive operators for search direction, analytical tools for structural diagnosis, and reflective memory for trajectory-level experience. Experiments on LLM-SRBench show that DE consistently outperforms representative LLM-based SR baselines across diverse scientific domains while using only 40% of the standard sample budget.

Problem

• symbolic regression
• sample efficiency
• LLM-based evolution
• search guidance
• scalar feedback

Innovation

• Deliberate Evolution
• Symbolic Regression
• Agentic Reasoning
• Sample Efficiency
• LLM-based Optimization