Neural Fidelity Calibration for Informative Sim-to-Real Adaptation

2025-04-11

AI Summary

To address poor policy generalization in simulation-to-reality (Sim2Real) transfer caused by physics model mismatch and perceptual uncertainty, this paper proposes the Online Neural Fidelity Calibration (NFC) framework. NFC introduces a novel conditional score-based diffusion model for dynamic parameter calibration, jointly modeling simulation-to-reality dynamics discrepancies via Bayesian residual estimation. It further integrates anomaly-triggered fine-tuning, sequential NFC priors, and optimistic hallucination–guided exploration to achieve information-efficient online adaptation. Evaluated in multi-robot high-dimensional parameter spaces, NFC achieves significantly higher calibration accuracy than state-of-the-art methods. Experimental results demonstrate substantial improvements in policy robustness: NFC enables a single-axle wheeled robot to maintain stable navigation on snow-covered terrain—a challenging real-world scenario where conventional Sim2Real approaches fail.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Intelligent Robots: State Estimation
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Deep reinforcement learning can seamlessly transfer agile locomotion and navigation skills from the simulator to real world. However, bridging the sim-to-real gap with domain randomization or adversarial methods often demands expert physics knowledge to ensure policy robustness. Even so, cutting-edge simulators may fall short of capturing every real-world detail, and the reconstructed environment may introduce errors due to various perception uncertainties. To address these challenges, we propose Neural Fidelity Calibration (NFC), a novel framework that employs conditional score-based diffusion models to calibrate simulator physical coefficients and residual fidelity domains online during robot execution. Specifically, the residual fidelity reflects the simulation model shift relative to the real-world dynamics and captures the uncertainty of the perceived environment, enabling us to sample realistic environments under the inferred distribution for policy fine-tuning. Our framework is informative and adaptive in three key ways: (a) we fine-tune the pretrained policy only under anomalous scenarios, (b) we build sequential NFC online with the pretrained NFC's proposal prior, reducing the diffusion model's training burden, and (c) when NFC uncertainty is high and may degrade policy improvement, we leverage optimistic exploration to enable hallucinated policy optimization. Our framework achieves superior simulator calibration precision compared to state-of-the-art methods across diverse robots with high-dimensional parametric spaces. We study the critical contribution of residual fidelity to policy improvement in simulation and real-world experiments. Notably, our approach demonstrates robust robot navigation under challenging real-world conditions, such as a broken wheel axle on snowy surfaces.

Problem

• Calibrates simulator physics for real-world adaptation
• Reduces sim-to-real gap with residual fidelity modeling
• Enables robust policy fine-tuning under uncertain conditions

Innovation

• Uses conditional score-based diffusion models
• Calibrates simulator coefficients online
• Optimistic exploration for policy optimization