Watch, Recall, Act: Always-On Robots in Concurrent Embodied Streams

2026-09-23

AI Summary

研究解决了始终在线的机器人在持续变化环境中执行任务的问题，通过ARMS系统整合实时感知、身体状态和过去动作来指导行动。

Abstract

An always-on robot faces an endless stream that never resets: instructions arrive and lapse, the scene changes, and its own past actions reshape what it must reason about. Today's action models are built for the opposite: a fixed instruction, no mid-task intervention, single-step reasoning. In an open-ended world a robot must watch a live stream for far-future cues, recall its own far-past actions, and act on them under dual-arm concurrency. We present ARMS (Always-on Robot in Multi-modal Streams), a deliberately simple streaming policy: a single pretrained $π$0.5 backbone augmented by three lightweight modules that turn live perception, embodied states, and the robot's own past actions into context the backbone reads before it acts. The modules update this context asynchronously, so watching and recalling never block acting and the two arms act at once. Rather than inventing new mechanisms, ARMS integrates these learned context providers with an agent-causal self-history that logs which arm did what, and when. To supervise them without extra annotation, we build ARMS Dataset, whose staged construction script itself labels every module from real dual-arm teleoperation. Trained on it, ARMS reaches 45% on the combined task against 28% for the strongest of our four main baselines, and ablations confirm the memory module, the embodied-state head, and asynchronous concurrency are each necessary.

Problem

• always-on robot
• continuous environment
• concurrent actions
• live stream processing
• long-term memory

Innovation

• always-on robot
• multi-modal streams
• asynchronous concurrency
• embodied states