Library Reachability in LSR-Synth: How Anti-Memorization Design Changes the Measurement of Symbolic Discovery

2026-07-29

AI Summary

This work addresses a critical limitation in existing benchmarks for scientific equation discovery, which often fail to distinguish whether models genuinely infer underlying laws or merely reproduce known formulas. To this end, the authors propose the LSR-Synth evaluation framework, which introduces novel synthetic terms into established mechanisms and incorporates strategies such as semantic blinding, operator library weakening, and exclusion of matching operator families to construct a semantics-free baseline. This setup rigorously isolates and quantifies the marginal contribution of language model priors. Experiments reveal that, under current task sets and search budgets, fixed operator libraries already cover most problems; only when lexical coverage is selectively disrupted do language model–generated candidate solutions substantially increase the number of solvable instances, thereby providing the first quantitative evidence of their practical value in out-of-distribution symbolic regression.

Abstract

Existing benchmarks for scientific equation discovery are largely composed of well-known equations available in the public domain, making it difficult to determine whether a model is discovering laws from data or merely recalling answers from its training corpus. LSR-Synth mitigates this problem by introducing novel synthetic terms into established scientific mechanisms and filtering the resulting tasks for novelty, solvability, and scientific plausibility. This paper examines a narrower measurement question: can these tasks further distinguish scientific priors supplied by language models from conventional operator search that does not access task semantics? We construct a semantics-free baseline using a fixed vocabulary with publicly documented provenance, and assess the role of candidate coverage through semantic blinding, library weakening, and matched operator-family knockouts. Under the current task snapshot, search budget, and scoring protocol, the fixed vocabulary already covers most tasks, while language-model-generated candidates rarely expand the set of solvable instances. Their marginal contribution becomes substantial only when vocabulary coverage is selectively disrupted. Strict out-of-distribution evaluation lowers the absolute success rates of all methods but does not alter this relationship. These findings neither invalidate LSR-Synth's controls against memorization of complete formulas nor imply that language-model priors are generally unhelpful. Rather, they support a more limited conclusion: most current tasks remain suitable for evaluating the fitting and recombination of previously unseen expressions, but are insufficient on their own to identify contributions from priors beyond a fixed search space.

Problem

• symbolic discovery
• memorization
• scientific equation discovery
• language model priors
• out-of-distribution evaluation

Innovation

• symbolic discovery
• anti-memorization
• language model priors
• fixed vocabulary baseline
• semantic blinding