ULTRADISCOVERY: Abductive Exploration in an Interconnected, Epistemically Open Universe

2026-10-02

AI Summary

This study addresses the challenge that scientific discovery requires agents to possess both representation construction and cross-domain evidence integration capabilities, yet existing benchmarks cannot independently control these factors. To overcome this limitation, this work constructs an interactive world environment employing a 2×2 experimental design to decouple representational openness from evidential distributionality for the first time. By integrating multi-domain interaction simulations, fixed latent dynamics designs, and large language model evaluation techniques, it systematically assesses agents' abductive exploration abilities. The primary contribution is the introduction of the first quantifiable benchmark for scientific discovery, which reveals fundamental limitations in current models regarding the incorporation of unobserved entities and variable rewriting. Furthermore, it precisely identifies the core bottleneck in transitioning from evidence accumulation to transferable representation construction.

Abstract

Scientific discovery often begins when scattered clues call for a new way of describing the world. Such abductive exploration can require constructing the representation in which an explanation is stated, when the world is epistemically open, and composing evidence scattered across contexts, when it is structurally interconnected. Existing benchmarks rarely separate these two demands or control them independently. We introduce ULTRADISCOVERY, an interactive world of five domains in which an agent revises an initially successful theory and predicts the outcome of an unseen cross-domain intervention. A $2 \times 2$ design leaves the representation open or discloses it, and leaves the evidence distributed or aligns it, with the latent dynamics fixed. With the representation open, agents across eleven models often retract the axiom they were taught, and none introduces the unobserved entity or rewrites the variables that a replacement requires. Disclosure triples intervention requests and adds about one of the eighteen findings the world affords, and alignment adds less. Two vendor-harness systems carry discovery into more domains, and one of them rewrites the variables in Open episodes. No system makes the exact prediction within 200 paid actions. At larger budgets one exact prediction appears with both aids, while every Open episode remains inexact. The results locate the difficulty in the step from accumulating evidence to composing it into a representation that transfers.

Problem

• abductive exploration
• scientific discovery
• epistemically open world
• cross-domain transfer
• representation construction

Innovation

• Abductive Exploration
• Scientific Discovery Benchmark
• Epistemic Openness
• Representation Construction
• Cross-domain Transfer