Imitation of Arm Gestures by the Semi-Humanoid Robot NICO

2026-07-20

AI Summary

This work proposes a method for driving a semi-humanoid robot to imitate human arm poses in real time using only monocular RGB video, aiming to enhance the naturalness of human–robot interaction. The system employs MediaPipe to estimate 3D human body keypoints, derives joint angles through analytical geometry, and maps these angles onto the NICO robot to reproduce corresponding motions. Requiring only an ordinary camera, this approach achieves semantically consistent pose imitation while significantly simplifying the perception pipeline. Experiments with six participants of varying heights demonstrate successful replication of diverse representative arm poses, confirming the feasibility of the proposed framework. However, the study also reveals current limitations in replicating highly complex postures and fine-grained wrist movements.

Abstract

Seamless human-robot interaction (HRI) requires a number of perceptual and motor abilities from the robot, one of them being the imitation of human gestures. Humanoid robots have an advantage in HRI thanks to their anthropomorphic features. In this work, we develop a system for imitation of human arm gestures by the semi-humanoid robot NICO based on analytical geometry and a pretrained MediaPipe pose-estimation model. For each input RGB frame, 3D coordinates of relevant human body landmarks, including arm joints and hand keypoints, are obtained using the MediaPipe framework. Joint angles are then computed from these coordinates using derived geometric relations. Finally, the computed angles are properly mapped to NICO's motor configuration and executed in a predefined motion sequence. Preliminary experiments on several representative arm gestures with six participants of different height indicate that the proposed method can produce meaningful imitative motions from monocular RGB input only, while also highlighting limitations in more complex poses and wrist-related movements.

Problem

• human-robot interaction
• gesture imitation
• semi-humanoid robot
• arm gestures
• pose estimation

Innovation

• gesture imitation
• human-robot interaction
• pose estimation
• analytical geometry
• semi-humanoid robot