BadWAM: When World-Action Models Dream Right but Act Wrong

2026-07-16

AI Summary

This work addresses the vulnerability of the alignment mechanism between imagination and action in World-Action Models (WAMs) to adversarial perturbations, which can severely degrade behavioral performance. The authors propose BadWAM, a framework that formally defines and implements two WAM-specific attack types: explicit attacks that solely disrupt actions, and stealthy attacks that preserve the realism of future predictions while inducing erroneous actions. By integrating visual adversarial perturbations, action optimization, and prediction regularization, BadWAM enables flexible control over attack intensity and stealthiness. Experimental results in closed-loop tasks demonstrate that such attacks substantially impair model performance—explicit action-targeted attacks reduce task success rates from 96.5% to 43.1%—thereby revealing an inherent fragility in the WAM alignment mechanism.

Abstract

World-action models (WAMs) are emerging as a promising foundation for embodied control: rather than predicting actions alone, they learn representations that couple action generation with future world prediction. This coupling is often viewed as a source of robustness, interpretability, and safety, as a robot's action can in principle be checked against its imagined future. In this paper, we show that this assumption is fragile. We introduce BadWAM, a unified framework for modeling and evaluating World-Action Drift Attacks: a new class of WAM-specific adversarial attacks that use small visual perturbations to break the alignment between what a WAM imagines and what it executes. BadWAM characterizes this attack surface along two natural criteria: attack strength and stealthiness. When the adversary prioritizes disruption, BadWAM instantiates an action-only adversarial attack, which directly drives the model toward task-failing actions. When the adversary additionally prioritizes stealth, BadWAM instantiates an imagination-preserving adversarial attack, which seeks to induce harmful action shifts while keeping the model's predicted future close to its clean imagination. Together, these two attacks capture a spectrum of WAM-specific failures: from overt action hijacking to stealthier cases where the model appears to imagine a plausible future but executes a desynchronized action. We evaluate BadWAM across different variants of WAMs. Results show that our attacks substantially reduce task success rates under closed-loop execution. For example, our action-only attack reduces the model performance from 96.5% to 43.1% success. The results of our imagination-preserving attack further exposes a WAM-specific vulnerability: moderate future-preserving regularization can maintain strong attack performance while reducing future imagination drift.

Problem

• World-Action Models
• Adversarial Attacks
• Action-Imagination Misalignment
• Embodied Control
• Model Robustness

Innovation

• World-Action Models
• adversarial attacks
• action imagination alignment
• stealthy perturbations
• embodied AI security