ActFovea: Runtime Safeguarding for VLA Policies via Spatiotemporal Visual-Action Consistency

2026-07-31

AI Summary

This work addresses the vulnerability of vision-language-action (VLA) policies to runtime disturbances that disrupt temporal alignment among visual observations, robot states, and actions, often leading to task failure. The authors propose ActFovea, a plug-and-play runtime safeguarding framework that, for the first time, leverages spatiotemporal vision-action consistency to detect and mitigate perturbations without requiring policy retraining. ActFovea constructs action-conditioned foveal regions by integrating robot kinematics, proprioception, and recent actions, then ensures robustness through consistency verification, perturbation-specific observation reconstruction, and a bounded safe-failure mechanism. Experiments on the LIBERO benchmark demonstrate that ActFovea improves task success rates under partial occlusion from 49.3% to 90.3%, yields gains of 7.0 and 9.8 percentage points under action drift and visual latency respectively, and achieves 100% safe failure in frozen-observation replay scenarios.

Abstract

Vision-language-action (VLA) policies achieve strong performance in robotic manipulation but remain vulnerable to runtime disturbances that break the temporal alignment among visual observations, robot states, and executed actions. We introduce ActFovea, a plug-and-play safeguarding framework that detects and mitigates such failures without retraining or modifying the underlying VLA policy. ActFovea uses robot kinematics, proprioceptive states, and recent actions to construct action-conditioned foveated regions that retain contact-relevant areas and predicted motion corridors while suppressing task-irrelevant visual content. It detects runtime risks by evaluating whether visual motion and observation freshness remain consistent with geometric, proprioceptive, and action transitions. For recoverable disturbances, ActFovea constructs disturbance-specific candidate observations and accepts a recovery only after verifying the resulting action chunk. When stale or replayed observations make reliable recovery impossible, it invokes a bounded safe-failure procedure. In closed-loop evaluations of $π_0$ across multiple LIBERO suites, ActFovea increases success under localized visual overlays from 49.3\% to 90.3\%, closing 93.7\% of the gap to clean performance. It further improves success under action drift and visual delay by 7.0 and 9.8 percentage points, respectively, while preserving clean-task performance. Under frozen-observation replay, ActFovea triggers timely safe failure in all trials, with no unprotected failures. These results demonstrate that spatiotemporal visual-action consistency provides an effective basis for runtime safeguarding of VLA policies.

Problem

• Vision-language-action
• runtime disturbances
• temporal alignment
• visual-action consistency
• robotic manipulation

Innovation

• visual-action consistency
• runtime safeguarding
• foveated perception
• VLA policies
• safe failure recovery