LiLa-WAM: Lightweight Latent Reasoning World-Action Model for Robotic Manipulation

2026-08-04

AI Summary

This work addresses the high computational cost, reliance on multi-stage training, and inclusion of control-irrelevant visual redundancy in existing World-Action Models (WAMs). To overcome these limitations, the authors propose LiLa-WAM, a lightweight architecture that jointly optimizes future state prediction and action generation within a compact latent space. Central to this approach is the introduction of language-agnostic Visual Transition Tokens (VTTs) as task representations, enabling fully end-to-end training on a single GPU. The method substantially reduces training costs while maintaining strong performance, achieving a 90.48% success rate across 50 tasks in RoboTwin 2.0, and demonstrating effectiveness on both LIBERO benchmarks and real-world robotic tasks—all trained on a single 24GB GPU.

Abstract

World-action modeling has emerged as a promising paradigm for robotic control, as it empowers models to go beyond reacting to observations and anticipate how a scene will evolve. However, existing WAMs often incur substantial computational overhead. Pixel-space methods often allocate substantial capacity to visual details that may not be directly relevant to control, while some latent-space methods require multi-stage training to construct the reasoning space. The resulting training cost can make such methods difficult to train under modest computational budgets. In this work, we propose LiLa-WAM, a lightweight world-action model that reasons about the future in a compact latent space and can be trained end-to-end on a single 24GB GPU. Its core design is a compact latent reasoning space jointly shaped by future-state prediction and action generation, which keeps the model lightweight while remaining well aligned with control. For task specification, we further propose the Visual Transition Token(VTT), a language-free task representation that encodes each task as a direction in visual feature space. Experiments on RoboTwin~2.0, LIBERO, and real-robot tasks demonstrate LiLa-WAM's effectiveness, achieving 90.48\% success across 50 RoboTwin tasks with single-GPU training.

Problem

• world-action modeling
• computational overhead
• latent reasoning
• robotic manipulation
• training cost

Innovation

• Lightweight latent reasoning
• World-action model
• End-to-end training
• Visual Transition Token
• Robotic manipulation