FutureRTC: Real-Time Robot Execution with Anticipatory-Conditioned Action Chunking

2026-07-27

AI Summary

This work addresses the discontinuity in visuomotor actions caused by misalignment between prediction and execution in asynchronous visual-language-action (VLA) policies. To resolve this, the authors propose FutureRTC, a framework that enables smooth, low-latency real-time control without modifying the original policy. FutureRTC predicts observations and states at the actual execution time through a state correction module, a motion-prior-guided visual representation prediction mechanism, and a policy consistency loss that aligns predicted context with actual inputs. Integrated with motion-aware feature propagation, forward state roll-out correction, and a plug-in compatible architecture, FutureRTC significantly improves task success rates in both simulation and real-world environments, yielding smoother trajectories, faster execution, and enhanced robustness to inference latency.

Abstract

Real-time deployment of Vision-Language-Action (VLA) policies necessitates asynchronous execution, wherein subsequent action chunks are computed concurrently with the execution of the current chunk, leading to prediction-execution misalignment and manifesting as inter-chunk discontinuities. Existing methods either superficially smooth chunk boundaries, require costly policy optimization, or exclusively forward-predict proprioceptive states yet neglect critical visual observations. In this paper, we propose \textbf{FutureRTC}, a plug-and-play adaptation framework that predicts execution-time observations and states for asynchronous VLA control without modifying the underlying policy. Specifically, FutureRTC features a state correction module to compensate for the discrepancy between rolled-forward and actual execution-time proprioceptive states and an observation prediction module that forecasts execution-time visual representations by leveraging robot motion as an explicit physical prior through motion-aware feature transport and reconstruction. Furthermore, we introduce a policy consistency loss to align the action chunks generated from predicted contexts with those produced under the expected execution-time inputs of the VLA policy. Extensive experiments across simulated and real-world environments demonstrate that FutureRTC achieves superior robustness to inference delays, resulting in smoother trajectories, faster execution, and consistently higher task success rates.

Problem

• Vision-Language-Action
• asynchronous execution
• prediction-execution misalignment
• inter-chunk discontinuities
• real-time robot control

Innovation

• anticipatory-conditioned action chunking
• asynchronous VLA control
• motion-aware feature transport
• state correction
• policy consistency loss