iMaC: Translating Actions into Motion and Contact Images for Embodied World Models

2026-06-08

AI Summary

This work addresses the limited expressiveness and poor generalization of traditional embodied world models that rely on low-dimensional, structured action vectors, particularly when applied to heterogeneous agents and complex physical interactions. To overcome these limitations, the authors propose iMaC, a novel framework that pioneers the use of raw visual images as native action representations, eliminating the need for predefined action spaces. iMaC employs a dual-branch architecture comprising an image-based action encoder and a dynamics predictor, which jointly learn action embeddings and conditionally model environmental dynamics to enable high-fidelity state prediction and closed-loop control. Evaluated on public benchmarks and real-world robotic tasks, iMaC significantly outperforms existing vector-action-based methods in prediction accuracy, task success rate, and cross-scenario generalization.

Abstract

Embodied world models have emerged as a pivotal paradigm for visual robotic decision-making and interactive environment simulation. However, conventional embodied frameworks rely on low-dimensional structured action vectors (e.g., joint angles and end-effector poses), which suffer from limited expressive capacity, poor generalization across diverse embodiments, and unnatural dynamic modeling for complex physical interactions. To address these limitations, this paper proposesiMac (Image as Action Control), a novel unified control paradigm that treats raw visual images as native action representations for embodied world models. Departing from traditional explicit kinematic action encoding, iMac formulates continuous visual manipulation as image-based action tokens, which inherently encapsulate spatial motion intentions, interactive geometric constraints and subtle physical dynamics. We construct a dual-branch embodied architecture consisting of an image-action encoder and a dynamic world predictor: the encoder compresses target-driven visual images into compact action embeddings, while the predictor learns environment transition rules conditioned on image actions to achieve high-fidelity future state prediction and closed-loop embodied control. Extensive experiments are conducted on public embodied manipulation benchmarks and real-world robotic scenarios. The results demonstrate that iMac outperforms vector-based action control baselines in prediction accuracy, task success rate and cross-scene generalization ability. Moreover, our image-action design eliminates the reliance on manually defined action spaces, realizing flexible and universal control for heterogeneous embodied agents. This work provides an innovative visual-action perspective for embodied world models, offering a simple yet effective paradigm for scalable robotic perception and manipulation.

Problem

• embodied world models
• action representation
• visual control
• generalization
• physical interaction

Innovation

• image-as-action
• embodied world models
• visual control
• action representation
• cross-embodiment generalization