Beyond Solution-Centric Search: Adaptive Inquiry and Knowledge Revision for Autonomous ML Engineering

2026-08-03

AI Summary

This work addresses the inefficiency in information acquisition and management inherent in long-horizon autonomous machine learning due to fixed solution structures. To overcome this limitation, the paper introduces an “information paradigm” that shifts away from traditional solution-centric search toward optimization guided by dynamically evolving information states. The proposed paradigm employs revisable task knowledge representations with explicit scopes and states, enabling multi-granularity decision contexts. Built upon large language models, it implements an inquiry-revision loop (Iris) that integrates local planning, cognitive probing, cross-experiment synthesis, and structured updates. Evaluated on MLE-Bench, the system achieves a state-of-the-art 64.9% any-medal rate within 12 hours and demonstrates strong generalization across four diverse out-of-domain tasks.

Abstract

Long-horizon autonomous research tasks such as machine learning engineering require systems to make interdependent decisions under a limited budget. Existing LLM-based agents typically organize candidate-solution improvement through tree, graph, or chain structures, meaning that the search process determines how information is acquired and managed. We call this design solution-centric search and propose instead the information paradigm, in which an evolving information state represents the system's understanding of the task and guides solution improvement. We instantiate this paradigm in Iris, an inquiry-revision loop. For information acquisition, Iris generates local action plans from the current information state and uses epistemic actions to probe decision-critical unknowns without modifying the retained solution. For information management, Iris synthesizes observations across experiments into task knowledge composed of revisable claims with explicit scope and status. It updates this knowledge as new evidence arrives and constructs each decision context from raw evidence, structured summaries, or task knowledge at the required level of detail. On MLE-Bench, Iris attains a 64.9% any-medal rate under a 12-hour budget, the highest among compared systems. Across four tasks spanning harness engineering and model post-training, Iris also demonstrates cross-domain generalization.

Problem

• autonomous ML engineering
• solution-centric search
• information acquisition
• knowledge revision
• long-horizon decision-making

Innovation

• information paradigm
• inquiry-revision loop
• epistemic actions
• revisable task knowledge
• autonomous ML engineering