Proprioceptive-visual correspondence enables self-other distinction in humanoid robots

2026-06-11

AI Summary

Humanoid robots struggle to develop social intelligence in cohabitative environments due to their inability to distinguish self from others. This work proposes an unsupervised method that leverages cross-modal correspondence between proprioceptive and visual signals to accurately differentiate self from others without requiring identity labels or predefined kinematic models. The approach constructs a dynamic, predictive 3D egocentric occupancy model of the robot’s own body. Notably, it achieves the first unsupervised self–other distinction in multi-agent settings. The resulting self-model effectively supports complex interactive tasks—including object grasping, collision-free motion planning, and human motion retargeting—thereby significantly enhancing the robot’s social cognition and autonomous interaction capabilities.

Abstract

Distinguishing self from others is a prerequisite for social intelligence, yet humanoid robots that increasingly share workspaces with humans still lack this ability. Here we show that a humanoid robot can learn self-other distinction from proprioceptive-visual correspondence, without any identity labels or kinematic models. Once established, this distinction bootstraps a predictive self-model that maps joint configurations to three-dimensional body occupancy, capturing how the robot's body changes with action. In multi-agent scenes involving humans or morphologically identical robots, the system reliably identifies itself, learns a 3D self-model, and supports downstream tasks including target reaching, collision-aware motion planning, and human-to-robot motion retargeting. Together, these results outline a route toward bodily self-representation in robots that act and coordinate alongside others in shared physical environments. Project page: https://euron-zc.github.io/humanoid-self-model/.

Problem

• self-other distinction
• humanoid robots
• proprioceptive-visual correspondence
• bodily self-representation
• social intelligence

Innovation

• proprioceptive-visual correspondence
• self-other distinction
• predictive self-model
• humanoid robots
• 3D body occupancy