Predictive Training with Latent Imagination for Visual Quadruped Navigation

2026-07-20

AI Summary

This work addresses the challenge of reactive navigation in quadrupedal robots operating in dynamic environments, where reliance solely on current observations often leads to delayed obstacle avoidance and collisions due to an inability to anticipate moving obstacles. To overcome this limitation, the authors propose a foresighted navigation method that incurs zero inference overhead by incorporating a lightweight predictive supervision signal during training. Leveraging a JEPA-style auxiliary predictor and SIGReg regularization, the approach guides the policy network’s hidden states to implicitly encode future scene dynamics without altering the inference-time controller architecture. Built upon an LSTM-SRU backbone within an end-to-end reinforcement learning framework, the method significantly improves navigation success rates and reduces collision frequency in both simulated and real-world dynamic environments, achieving zero-shot sim-to-real transfer on the Unitree Go2 platform without fine-tuning.

Abstract

Reinforcement-learning navigation policies for legged robots select actions reactively from current observations and short-term memory, with limited capacity to anticipate how moving obstacles will evolve in the near future. In dynamic environments, this reactivity causes the robot to respond too late because collision risk depends on short-horizon scene structure rather than on current obstacle positions alone. Lightweight predictive supervision applied to the policy's recurrent state during training can encode anticipatory obstacle dynamics without modifying the inference-time controller. We augment a reactive LSTM-SRU navigation backbone with an auxiliary JEPA-style predictor and SIGReg regularization: during training, the predictor supervises the deterministic hidden state to anticipate its own next state; at inference, it is fully discarded, incurring zero additional computational cost. On simulated and real-world navigation benchmarks with dynamic obstacles, our method substantially improves navigation success while reducing collision rates through the predictive training signal alone, without additional inference-time parameters. Real-robot deployment on a Unitree Go2 demonstrates zero-shot sim-to-real transfer: the controller navigates cluttered indoor and dynamic outdoor environments without fine-tuning, with evasive behavior consistent with the collision reduction observed in simulation.

Problem

• visual quadruped navigation
• dynamic obstacles
• predictive training
• collision avoidance
• reactive policy

Innovation

• predictive training
• latent imagination
• JEPA-style predictor
• zero-shot sim-to-real
• dynamic obstacle navigation