KinematicRL: A Sim-to-Real Reinforcement Learning Framework For Social Navigation With Kinodynamic Feasibility

2026-06-10

AI Summary

This work addresses the sim-to-real transfer challenges in social navigation for deep reinforcement learning, which stem from oversimplified dynamics models and reliance on complex human state estimation. To overcome these limitations, the authors propose a unified framework that employs second-order control inputs to model higher-order robot dynamics and leverages only 2D LiDAR for cluster-based pedestrian tracking. A novel unbiased residual gating mechanism is introduced to balance reactive and memory-driven behaviors. The approach integrates stochastic iterative LQR pretraining with deep reinforcement learning and is successfully deployed on a real differential-drive robot. Experimental results demonstrate significantly improved kinematic feasibility, strong robustness, and adaptability to dynamically varying crowd sizes.

Abstract

Deep Reinforcement Learning (DRL) has shown promise for social navigation, yet its real-world deployment remains hindered by a persistent sim-to-real gap arising from simplified first-order dynamics and context-specific human state estimation pipelines. This work presents a unified framework that addresses these limitations to produce dynamically feasible navigation policies suitable for real-world deployment. First, theoretical analysis reveals that tracking error between simulated and actual robot position decays exponentially with increased control order, motivating the use of higher-order control inputs as DRL action space. A second-order control formulation tailored to differential drive robots is developed, complemented by a stochastic iterative Linear Quadratic Regulator (iLQR) that pretrains the policy via a divergence minimization objective. Second, to avoid the added system complexity of camera-LiDAR fusion, a cluster-based human tracking pipeline using only 2D LiDAR is introduced. Human detections are associated according to both spatial proximity and velocity similarity, enabling reliable differentiation of nearby pedestrians and yielding stable velocity estimates through temporal aggregation. Third, we introduce an unbiased residual gating block to balance reaction- and memory-based behaviors while handling time-varying crowd sizes, both critical for social navigation. The resulting policy, KinematicRL, consistently improves kinematic performance and adapts to varying number of detected humans. Experiments in real-world environments demonstrate that, when combined with the proposed tracking pipeline, KinematicRL can be deployed on a real differential drive robot with minimal modifications.

Problem

• sim-to-real gap
• social navigation
• kinodynamic feasibility
• human state estimation
• reinforcement learning

Innovation

• kinodynamic feasibility
• sim-to-real reinforcement learning
• second-order control
• LiDAR-based human tracking
• residual gating