X-NavDP: Generalizing Navigation Diffusion Policy to Novel Behavior and Embodiments with Group Q-score Reweighted Matching

2026-07-30

AI Summary

This work addresses the limited generalization of existing pre-trained navigation diffusion policies, which rely on expert data generated by a single idealized planner and struggle to adapt to new robot morphologies and complex environments. To overcome this, the authors propose the GQRM framework, which innovatively integrates behavior-perturbation-based bootstrap exploration with a group-wise Q-score normalization mechanism. This enables efficient reweighted score matching that preserves pre-trained priors while supporting distributed online reinforcement learning fine-tuning across heterogeneous robot embodiments. The approach substantially improves both generalization and training stability for cross-embodiment visual navigation, achieving state-of-the-art performance: success rates increase from 61.20% to 84.28% in simulation and from 10% to 65% in challenging real-world scenarios.

Abstract

Pretraining navigation diffusion policies rely on large-scale expert demonstrations. These data are typically generated by a fully-informed oracle planner suited to a single nominal robot. This limits the policy's generalization to diverse embodiments and challenging scenarios (e.g., escaping dead ends or detouring long obstacles) that demand diverse local reactive behaviors with only onboard local observations. Post-training the policy with reinforcement learning (RL) offers a principled remedy. However, previous RL for diffusion approaches lead to only marginal improvements. This is because the intractable likelihood of diffusion policies renders policy gradients unstable in addition to inefficient policy exploration. To address these challenges, we propose a data-efficient diffusion RL post-training framework - GQRM (Group Q-score Reweighted Matching). Our framework introduces two complementary designs: (i) a self-bootstrapped exploration strategy with behavior perturbation that preserves the pretrained policy prior, and (ii) a group Q-score normalization mechanism that computes per-trajectory values on each state for efficient reweighted score matching. By conducting distributed online RL training across heterogeneous embodiments, the resulting fine-tuned policy, X-NavDP, achieves state-of-the-art cross-embodiment visual navigation performance, improving the overall success rate from 61.20% to 84.28% in simulation and 10% to 65% in real-world hard cases. The code and model are publicly available at https://yty-sky.github.io/x-navdp-project-page.

Problem

• navigation diffusion policy
• embodiment generalization
• local reactive behavior
• visual navigation
• expert demonstrations

Innovation

• Diffusion Policy
• Reinforcement Learning
• Cross-embodiment Generalization
• Q-score Reweighting
• Visual Navigation