Problem
Research questions and friction points this paper is trying to address.
self-evolving search
imperfect proxy rewards
false positives
large language models
reliable scientific discovery
Reliable Self-Evolution with Imperfect Proxy Rewards
🤖 AI Summary
This study addresses the false-positive contamination problem in large language model self-evolutionary search caused by imperfect proxy rewards. To mitigate this issue, we propose CISE, a method that integrates conditional conformal inference with online density ratio estimation to construct statistically calibrated, conservative reward intervals. A filtering mechanism is further introduced to ensure reliable coverage throughout the iterative process, thereby achieving robust self-evolution. Evaluated on materials science tasks, CISE yields exclusively true-positive candidates, significantly enhancing search precision and output reliability under limited verification budgets.
📝 Abstract
Large language model (LLM)-based self-evolving search is a promising approach to scientific discovery. However, high-fidelity evaluation of every candidate is prohibitively expensive in some domains. Self-evolving systems in such settings therefore rely on low-cost but imperfect proxy rewards, which may assign high scores to infeasible candidates. These false positives may contaminate both the final output and the feedback used to guide subsequent generations. This motivates statistically calibrated reward intervals for more reliable self-evolving search. We propose Conformal Interval-Driven Self-Evolution (CISE), which constructs candidate-specific reward intervals using conditional conformal inference and iteration-wise online density-ratio estimation. CISE uses conservative interval-based rewards for evolutionary feedback and returns candidates only when all required property intervals lie entirely within their respective feasible regions. We derive fixed-iteration coverage results under explicit assumptions of independence and covariate shift. We evaluate CISE on three self-evolving search tasks in materials science. In our experiments, all candidates returned by CISE are true positives under high-fidelity evaluation, whereas the baselines return more candidates but include false positives. These results highlight the value of a smaller, more precise shortlist when downstream validation budgets are limited. Our repository is available at https://github.com/MLAI-Yonsei/CISE.git.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Conformal Inference
Self-Evolving Search
Proxy Rewards
Online Density-Ratio Estimation
Large Language Models
🔎 Similar Papers
No similar papers found.
