Learning to Generate Pointing Gestures in Situated Embodied Conversational Agents

2025-09-15

AI Summary

This work addresses the problem of inaccurate and unnatural gesturing by embodied agents (e.g., robots) during natural deictic communication with humans in physical environments. We propose a gesture generation framework that integrates imitation learning (IL) with hierarchical reinforcement learning (HRL), jointly modeling motion control policies and referential semantics using only small-scale motion-capture data—thus ensuring both physical plausibility and deictic precision. In human-subject evaluations within a virtual reality deictic task, our method significantly outperforms purely supervised baselines, improving referential accuracy by +12.3% and achieving statistically significant gains in perceived naturalness (p < 0.01). It has been successfully deployed on a real robotic platform, demonstrating practical viability. The core contribution is the first integration of hierarchical RL into an IL pipeline, enabling high-fidelity deictic gesture synthesis under low-data regimes.

Technology Category

• Machine Learning: Imitation Learning & Inverse Reinforcement Learning
• Intelligent Robots: Embodied AI
• Natural Language Processing: Code Generation / Program Synthesis from Natural Language

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

One of the main goals of robotics and intelligent agent research is to enable natural communication with humans in physically situated settings. While recent work has focused on verbal modes such as language and speech, non-verbal communication is crucial for flexible interaction. We present a framework for generating pointing gestures in embodied agents by combining imitation and reinforcement learning. Using a small motion capture dataset, our method learns a motor control policy that produces physically valid, naturalistic gestures with high referential accuracy. We evaluate the approach against supervised learning and retrieval baselines in both objective metrics and a virtual reality referential game with human users. Results show that our system achieves higher naturalness and accuracy than state-of-the-art supervised models, highlighting the promise of imitation-RL for communicative gesture generation and its potential application to robots.

Problem

• Generating natural pointing gestures for embodied agents
• Combining imitation and reinforcement learning for gesture control
• Improving referential accuracy and naturalness in non-verbal communication

Innovation

• Combining imitation and reinforcement learning
• Learning motor control policy for gestures
• Producing physically valid naturalistic gestures