Benchmarking Zero-Shot LLM-Generated Parent Selection in Genetic Programming for Symbolic Regression

2026-07-26

AI Summary

This study systematically investigates, for the first time, whether large language models (LLMs) can generate effective genetic programming parent selection operators for symbolic regression without any in-context examples or iterative feedback. Using a standardized natural language prompt, eight LLMs were prompted to produce selection operators, which were then evaluated on twelve OpenML regression benchmarks while keeping all other framework components and search budgets constant. The results demonstrate that LLMs can synthesize high-performing, semantically aware selection operators that outperform conventional lexicase and tournament selection. Among them, Kimi K2.5 achieved the best performance, significantly surpassing baseline methods, while Claude Sonnet 4.6 and Gemini 3.1 Pro exhibited robust generalization. The findings reveal that LLMs can autonomously construct non-trivial search heuristics from task descriptions alone, and that public LLM rankings strongly correlate with training R² but only weakly with test R².

Abstract

Parent selection significantly affects exploration, exploitation, and complexity control in genetic programming (GP) for symbolic regression. It is unclear whether large language models (LLMs) can synthesize effective operators in a zero-shot setting without iterative meta-evolution. Here, zero-shot means that the model receives only the task description, with no reference operators or iterative feedback. In this work, we benchmark zero-shot synthesis of parent-selection operators across eight LLMs within a standard GP framework for symbolic regression. Each model receives the same natural-language prompt to generate a parent-selection operator, which is then evaluated in a standard GP framework with only the parent-selection operator replaced, while all other components and the evolutionary-search budget are held constant. For each LLM, ten independent zero-shot operators are evaluated on twelve OpenML regression benchmarks and compared against automatic lexicase and tournament selection baselines. Claude Sonnet~4.6 and Gemini~3.1 Pro stand out for consistently strong performance on both training and held-out test $R^2$. The strongest operator in our benchmark---a Kimi~K2.5 zero-shot synthesis---surpasses the automatic lexicase and tournament baselines in search effectiveness. These results suggest that zero-shot LLM synthesis is a viable approach to generating competitive GP selection operators. Analysis shows that many generated operators use semantics to guide selection, suggesting that LLMs can produce non-trivial search heuristics from the task description alone. We also examine the relationship between public LLM leaderboard rankings and GP performance. Widely used benchmarks, such as Humanity's Last Exam and SWE-bench Verified, strongly correlate with training $R^2$, while their relationship to held-out test $R^2$ is weaker and less clear.

Problem

• zero-shot
• large language models
• genetic programming
• symbolic regression
• parent selection

Innovation

• zero-shot LLM synthesis
• genetic programming
• parent selection
• symbolic regression
• search heuristics