Simulation Distillation: Pretraining World Models in Simulation for Rapid Real-World Adaptation

2026-03-16

Technology Category

Machine Learning: Imitation Learning & Inverse Reinforcement LearningSearch and Optimization: Sampling/Simulation-based SearchPlanning, Routing, and Scheduling: Replanning and Plan Repair

Application Category

User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systemsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingWeb Mining and Content Analysis: Web data generation and simulation

Abstract

Simulation-to-real transfer remains a central challenge in robotics, as mismatches between simulated and real-world dynamics often lead to failures. While reinforcement learning offers a principled mechanism for adaptation, existing sim-to-real finetuning methods struggle with exploration and long-horizon credit assignment in the low-data regimes typical of real-world robotics. We introduce Simulation Distillation (SimDist), a sim-to-real framework that distills structural priors from a simulator into a latent world model and enables rapid real-world adaptation via online planning and supervised dynamics finetuning. By transferring reward and value models directly from simulation, SimDist provides dense planning signals from raw perception without requiring value learning during deployment. As a result, real-world adaptation reduces to short-horizon system identification, avoiding long-horizon credit assignment and enabling fast, stable improvement. Across precise manipulation and quadruped locomotion tasks, SimDist substantially outperforms prior methods in data efficiency, stability, and final performance. Project website and code: https://sim-dist.github.io/

Similar Papers

CityBench: Evaluating the Capabilities of Large Language Models for Urban Tasks

2024-06-20Citations: 0