OminiAdapt: Learning Cross-Task Invariance for Robust and Environment-Aware Robotic Manipulation

2025-03-27

AI Summary

To address covariate shift in egocentric visual imitation learning for humanoid robots—arising from complex perception–control coupling and morphological/actuation disparities between humans and robots—this paper proposes a task-goal-centric cross-task invariance learning paradigm. We introduce a novel dynamic weight update mechanism that jointly leverages channel-wise feature fusion and spatial attention, integrating egocentric visual representation learning, multimodal feature alignment, and adaptive attention optimization. Evaluated on fine manipulation tasks—including grasping, peg insertion, and screw turning—the method achieves an average success rate improvement of 23.6%, demonstrating significantly enhanced robustness across diverse environments and superior generalization to unseen tasks. The approach is highly scalable and modular, with all code and benchmark datasets publicly released.

Technology Category

• Intelligent Robots: Manipulation
• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Computer Vision: Multi-modal Vision

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

With the rapid development of embodied intelligence, leveraging large-scale human data for high-level imitation learning on humanoid robots has become a focal point of interest in both academia and industry. However, applying humanoid robots to precision operation domains remains challenging due to the complexities they face in perception and control processes, the long-standing physical differences in morphology and actuation mechanisms between humanoid robots and humans, and the lack of task-relevant features obtained from egocentric vision. To address the issue of covariate shift in imitation learning, this paper proposes an imitation learning algorithm tailored for humanoid robots. By focusing on the primary task objectives, filtering out background information, and incorporating channel feature fusion with spatial attention mechanisms, the proposed algorithm suppresses environmental disturbances and utilizes a dynamic weight update strategy to significantly improve the success rate of humanoid robots in accomplishing target tasks. Experimental results demonstrate that the proposed method exhibits robustness and scalability across various typical task scenarios, providing new ideas and approaches for autonomous learning and control in humanoid robots. The project will be open-sourced on GitHub.

Problem

• Address covariate shift in humanoid robot imitation learning
• Improve robustness against environmental disturbances in manipulation tasks
• Bridge physical differences between humans and humanoid robots

Innovation

• Cross-task invariance learning for robust manipulation
• Channel feature fusion with spatial attention
• Dynamic weight update strategy for task success