ActiveMimic: Egocentric Video Pretraining with Active Perception

2026-06-04

AI Summary

This work addresses the limitations of existing methods that pretrain robot models using human first-person videos, which often underperform due to their neglect of active perception behaviors—such as viewpoint adjustments—inherent in such videos. The paper introduces a novel approach that explicitly models camera motion as “viewpoint actions,” recovering synchronized camera and wrist trajectories from monocular body-worn RGB video. It further proposes a unified pretraining framework that jointly learns active perception and manipulation behaviors. Through cross-domain transfer, the method significantly outperforms human-video-based baselines on multiple real-world tasks requiring active perception and achieves performance on par with state-of-the-art models pretrained on robot-collected data.

Abstract

Egocentric human video offers a scalable alternative to robot data for pretraining, yet models pretrained on such video consistently underperform those pretrained on robot data. We attribute this gap to a missing signal, the active perception behavior in egocentric videos, where humans continuously reposition their viewpoint during manipulation, inducing camera motion that standard pipelines treat as noise. To address this, we present ActiveMimic, a pretraining framework that recovers synchronized camera and wrist trajectories from a single body-worn RGB camera, models camera motion as a viewpoint action, and jointly learns active perception and manipulation from in-the-wild egocentric human video before adapting to a target robot. Empirically, real-world experiments across tasks with diverse active perception demands show that ActiveMimic consistently surpasses baselines pretrained on human video and matches state-of-the-art models pretrained on robot data. Further analysis provides evidence that active perception capability originates from egocentric human video pretraining rather than robot-specific fine-tuning, confirming active perception as the key to unlocking egocentric human video for robot pretraining.

Problem

• egocentric video
• active perception
• robot pretraining
• camera motion
• viewpoint action

Innovation

• active perception
• egocentric video
• pretraining
• camera motion modeling
• robot learning