AMBIT: Anticipatory Multimodal Body Recruitment for Bimanual Tracking on a Humanoid

2026-09-27

AI Summary

This study addresses the non-uniqueness of whole-body policies caused by redundant degrees of freedom and the mode-averaging tendency of deterministic models in humanoid dual-arm tracking. To overcome these limitations, a “generate-select-commit” architecture is proposed. Specifically, a conditional variational autoencoder (CVAE) generates diverse torso-waist strategies, which are validated by a non-learned selector and integrated with receding horizon control to achieve whole-body coordination. By preserving policy diversity, this approach enables zero-shot adaptation to unknown constraints, effectively circumventing the drawbacks of conventional regression. Evaluations in MuJoCo simulations and on a real Unitree robot demonstrate an 85% success rate and a median end-effector error of only 11 mm in complex dynamic environments, significantly outperforming baselines while confirming kinematic feasibility.

Abstract

A humanoid with 5-DoF arms cannot track generic bimanual end-effector trajectories with its arms alone; pelvis and waist motion must be recruited, but which motion, and when, is not uniquely determined. On a Unitree R1 in fixed double support, the set of dynamically valid recruitment strategies (pelvis pose and waist trajectories) for a task is a diverse continuous manifold, and a deterministic regressor trained on it mode-averages into strategies valid only 35% of the time, against 52% for a conditional variational autoencoder (CVAE) and 82% for the best of 16 CVAE samples. We introduce AMBIT: the CVAE proposes strategies from a preview of the commanded trajectory, a non-learned selector filters, ranks and verifies them, and a receding-horizon loop commits to one with hysteresis. The committed strategy is the reference of the same whole-body differential-IK QP a reactive tracker runs, which keeps authority over residual error. On 160 held-out episodes that admit a valid strategy, in full MuJoCo dynamics under a torque controller, AMBIT reaches 85% success at a 3 cm/15 deg tolerance against 74% for the tracker (disjoint confidence intervals) and recruits the body before the arms saturate in 48% of episodes against 35%. Because diversity is preserved, constraints unknown at training time are enforced by selection alone: under five zero-shot shifts AMBIT beats the warm-started tracker on every shift and matches a test-time re-optimisation baseline 17x more expensive. On a Unitree G1, with hyperparameters unchanged, the protocol reproduces the structure of the valid set and widens the gap over the tracker to 0.85 against 0.53. Five selected strategies execute on the externally supported physical R1, distinct in pelvis excursion and tracking the planned end-effector motion to a median of 11 mm by encoder forward kinematics, which establishes kinematic realisability, not balance.

Problem

• humanoid robot
• bimanual tracking
• body recruitment
• redundancy resolution
• multimodal strategy

Innovation

• Conditional Variational Autoencoder
• Receding-horizon Control
• Whole-body Differential IK
• Zero-shot Generalization
• Bimanual Tracking