One Hand Watches The Other: Dynamic Multi-Agent Cooperation for Sample-Efficient Bimanual Manipulation in Dynamic Environments

2026-07-24

AI Summary

Existing multi-stream strategies based on environmental reference frames fail in dynamic scenes because they assume the reference frame is entirely exogenous, making them ill-suited for tightly coupled interactions such as bimanual coordination or manipulating moving objects. To address this limitation, this work proposes DynaMAC, a novel framework that, for the first time, models the contralateral arm as a dynamic task parameter, enabling a unified representation of dynamic manipulation and bimanual collaboration without predefining master-slave roles and while remaining compatible with arbitrary policy architectures. DynaMAC employs a lightweight, policy-agnostic multi-stream architecture combined with dynamic task parameterization, achieving zero-shot generalization from static demonstrations to dynamic environments and substantially reducing data collection costs. Evaluated on the newly introduced DynaBench benchmark, DynaMAC outperforms state-of-the-art probabilistic and generative baselines by over 35 percentage points in dynamic manipulation and bimanual tasks while requiring 20 times fewer training samples.

Abstract

Multi-stream robot manipulation policies achieve unparalleled sample efficiency and generalization by modeling actions relative to environmental reference frames. However, existing approaches typically assume these frames to be strictly exogenous. This causal assumption collapses in dynamic settings, such as when a single robot arm manipulates a moving object or when two arms coordinate, where each arm effectively becomes part of the dynamic environment of the other. We propose DynaMAC, a lightweight, policy-agnostic framework that resolves this causal limitation while preserving the sample efficiency, computational speed, and flexibility of multi-stream policies, DynaMAC treats the opposite arm as a dynamic task parameter, thereby providing a unified formulation for dynamic manipulation and bimanual coordination without requiring an explicit leader-follower relationship. To rigorously evaluate these capabilities, we introduce DynaBench, a novel benchmark for robot manipulation in dynamic environments. Across both dynamic environments and bimanual manipulation tasks, DynaMAC outperforms leading probabilistic and generative baselines by over 35 percentage points while requiring 20 times fewer samples. Crucially, DynaMAC generalizes zero-shot from static demonstrations to dynamic environments, substantially simplifying data collection and establishing an elegant bridge toward human-robot collaboration.

Problem

• bimanual manipulation
• dynamic environments
• multi-agent cooperation
• sample efficiency
• reference frames

Innovation

• DynaMAC
• bimanual manipulation
• dynamic environments
• sample efficiency
• multi-agent cooperation