Scouting the Dynamics Gap: Test-Time Policy Adaptation via Action-Outcome Feedback

2026-09-28

AI Summary

This study addresses the difficulty of pretrained robot policies in adapting to unknown physical environments due to their reliance on sparse rewards and limited use of geometric dynamic feedback. To overcome this, we propose SCOUT, a framework that couples action prediction with forward dynamics models to construct a shared belief latent space. By leveraging dynamics-aware meta-learning, it establishes a bilevel architecture comprising an inner loop for belief updating and an outer loop for action optimization. The core innovation lies in utilizing action-outcome feedback to update internal dynamics beliefs online, enabling rapid and robust adaptation without sparse rewards while avoiding catastrophic forgetting. Experiments demonstrate that SCOUT significantly accelerates online adaptation in simulated manipulation benchmarks and successfully validates robust sim-to-real transfer capabilities.

Abstract

While pretrained robotic policies exhibit impressive capabilities in controlled environments, unobserved physical properties and dynamics require these policies to rapidly adapt during deployment. Existing test-time adaptation methods typically rely on sparse scalar rewards, failing to exploit the rich geometric and dynamic feedback from the environment during physical interaction. To address this challenge, we propose SCOUT, a dynamics-aware meta-learning framework that enables manipulation policies to rapidly adapt by continuously revising their internal beliefs about environment dynamics. Our approach couples an action-prediction policy with a forward dynamics model via a shared belief latent space. During meta-training, an inner loop updates this shared belief latent by minimizing the dynamics prediction error against the observed action outcome, while the outer loop optimizes the network for action selection. At deployment, this structure allows the agent to infer and adapt to unknown physical dynamics on the fly. By updating its latent belief based on action-outcome mismatches, the policy automatically adapts without risking catastrophic forgetting. We demonstrate that SCOUT significantly accelerates online adaptation across simulated manipulation benchmarks and achieves robust sim-to-real transfer in the real world. Project webiste can be found here: https://liy1shu.github.io/SCOUT/

Problem

• test-time adaptation
• robotic manipulation
• dynamics gap
• action-outcome feedback
• sim-to-real transfer

Innovation

• Test-Time Adaptation
• Meta-Learning
• Forward Dynamics Model
• Belief Latent Space
• Sim-to-Real Transfer