OPERA: Operator-residual feedback for reliable autonomous optical experiments with language-model agents

2026-08-06

AI Summary

This work addresses the limitation of existing autonomous agents in optical experimentation, whose scoring metrics often fail to faithfully capture physical outcomes, leading to ineffective or detrimental decisions. To overcome this, the authors propose the OPERA framework, which introduces—for the first time—an operator-residual feedback mechanism. Experimental actions are modeled as optical operators, and physically interpretable residuals quantify the deviation between actual outcomes and desired states, explicitly decoupling executable actions from physical state errors. Integrating language model agents, optical operator representations, and digital twin technology, OPERA enables closed-loop autonomous control grounded in measurable physical evidence. Evaluated across three optical tasks, the framework reduces the proportion of invalid decisions from 23.6–39.0% to 0.9–1.9%, substantially improving task success rates, stability, and the efficiency of protocol transfer and reconstruction on real instruments.

Abstract

Autonomous agents choose actions using scores that may not reflect experimental success. We developed OPERA, an operator-residual framework for optical experiments. It represents experimental actions as optical operators and evaluates their outcomes using physically interpretable residuals. Operators specify executable changes to measurement, control or reconstruction, while residuals report departures from specified physical conditions. The agent uses both to select, combine or generate operators, and physical performance is evaluated independently against a withheld reference. Across three optical tasks, score-only feedback produced score increases without physical improvement in 23.6--39.0\% of decisions, compared with 0.9--1.9\% for operator-residual feedback. Operator-residual feedback increased the probability of reaching and maintaining task targets and reduced experimental budgets. Protocols selected in digital twins were transferred to three optical instruments, and repeated experiments showed a lower projection budget in structured-light reconstruction. Together, operators and residuals guide autonomous decisions using measurable physical evidence.

Problem

• autonomous agents
• optical experiments
• feedback reliability
• physical performance
• operator-residual

Innovation

• operator-residual feedback
• autonomous optical experiments
• language-model agents
• physically interpretable residuals
• optical operators