Performance-Preserving Online Adaptation in Social Navigation via Diffusion Steering

2026-09-21

AI Summary

本文提出一种基于强化学习的扩散转向方法，通过仅训练噪声策略来保持基础模型性能，解决社交导航中机器人与人类复杂互动建模难题。

Abstract

In social navigation, modeling the complex interactions between humans and robots is difficult, and deep reinforcement learning has therefore been actively studied. However, because simulation alone cannot fully reproduce diverse scenarios, robot dynamics, and the social conventions that vary across deployment environments, fine-tuning in the deployment environment is promising. In doing so, learning that preserves the base model's performance is required, so as not to compromise the primary objective of navigation, namely avoiding pedestrians and reaching the destination. In this study, we propose a method that applies diffusion steering via reinforcement learning (DSRL), which trains only the noise policy while keeping the diffusion policy fixed, thereby achieving learning that preserves performance. Furthermore, we integrate diffusion-based RL policies trained with multiple seeds to construct the base policy, improving learning performance. Our evaluation shows that, compared with other methods, the proposed method enables efficient learning while preserving performance, and we confirm flexible behavior control through adaptation to social conventions, as well as its effectiveness on a physical robot through hardware-in-the-loop simulation.

Problem

• social navigation
• deep reinforcement learning
• deployment environment
• performance preservation
• social conventions

Innovation

• diffusion steering
• reinforcement learning
• performance-preserving learning
• social navigation