Retriever: Composing Closed-Loop Asynchronous Robot Programs

2026-07-19

AI Summary

This work addresses the challenges of irreproducibility, debugging difficulty, and limited reusability in long-horizon robotic systems caused by asynchronous execution of heterogeneous modules. To this end, the paper introduces Retriever—a full-stack framework that unifies the modeling of asynchronous closed-loop agents at both algorithmic and system levels for the first time. Retriever represents agents as stateful causal flow function graphs over an explicit runtime clock and formalizes their interaction with the environment as an asynchronous loop in continuous time. The framework enables deterministic orchestration, systematic cross-environment debugging, and high-fidelity log-based replay. Experimental results demonstrate that Retriever incurs low runtime overhead in real-world robotic tasks while effectively ensuring behavioral reproducibility and reusability.

Abstract

Building long-horizon robot agents requires composing closed-loop pipelines -- perception, belief update, planning, and control -- whose components run at different clocks and with variable latency. Today, these systems are often assembled with ad-hoc concurrency and pub/sub conventions that make timing and input-consumption semantics implicit, yielding schedule-dependent behavior that is hard to reproduce, debug, and reuse. Current solutions typically solve parts of this problem at either the algorithmic or the systems layer, but not both. In this work, we propose Retriever, which spans the entire stack: an asynchronous decision model, a programming model, a runtime, and an example closed-loop agent pipeline. Retriever represents an agent as a graph of stateful causal stream functions executed on explicit run clocks. We formalize this view via an asynchronous environment-agent loop over continuous-time streams and show that finite-memory causal policies can be represented by compositions of these operators. Retriever compiles these graphs into a runtime that supports multiple backends, enabling systematic debugging across running environments and deterministic replay from logged asynchronous data. We evaluate Retriever through a real-robot case study together with controlled studies of runtime overhead and deterministic replay behavior.

Problem

• asynchronous robot programs
• closed-loop systems
• timing semantics
• deterministic replay
• composable pipelines

Innovation

• asynchronous robot programming
• causal stream functions
• closed-loop agent
• deterministic replay
• stateful computation graph