ABot-World-0: Infinite Interactive World Rollout on a Single Desktop GPU

2026-07-21

AI Summary

This work addresses the challenge of achieving real-time, long-horizon, closed-loop controllable video generation on a single desktop GPU. It proposes an action-conditioned video world model trained on diverse data sources—including AAA games, simulations, and web videos—leveraging agent-driven data collection and unified annotation. The approach introduces LongForcing to align long-sequence self-evolution with a horizon-extended teacher model, while incorporating reference character memory to preserve identity consistency. A low-memory, high-frame-rate streaming architecture is realized through a lightweight VAE decoder, ODE distillation, efficient attention mechanisms, memory-aware scheduling, and low-bit DiT inference. On a single RTX 5090, the system achieves 720p@16 FPS output with a 1.2-second latency from action input to first frame and a peak memory footprint of 19 GiB, demonstrating strong controllability and long-term coherence.

Abstract

We present ABot-World-0, an action-conditioned video world model for real-time, long-horizon closed-loop interaction, supported by a multi-source data infrastructure spanning AAA games, simulation engines, and internet videos to learn controllable world dynamics. WorldExplorer performs agent-driven collection guided by training feedback, while a unified pipeline applies 14 deterministic quality checks, VLM-based assessment, and synchronized action and text annotation. We progressively distill a bidirectional action-conditioned teacher into a causal student through teacher forcing and ODE distillation, and introduce LongForcing to align long student self-rollouts with an extended-horizon teacher, mitigating accumulated distribution shift and autoregressive drift. Raw keyboard actions provide a unified control interface for scene roaming and third-person character interaction, while reference-character memory provides persistent appearance cues for identity consistency during third-person rollouts. For deployment, we co-design a streaming inference stack with a lightweight VAE decoder, efficient attention, memory-aware scheduling, and low-bit DiT inference. Across optimized low-bit configurations, ABot-World-0 streams 720P video at up to 16 FPS on a single NVIDIA RTX 5090 desktop GPU, with 1.2s action-to-first-frame latency and approximately 19GiB peak VRAM. Experiments on WorldRoamBench and extended interactive rollouts demonstrate competitive controllability and coherent long-horizon world evolution.

Problem

• interactive world rollout
• action-conditioned video generation
• long-horizon controllability
• real-time world modeling
• single-GPU deployment

Innovation

• action-conditioned world model
• LongForcing
• teacher-student distillation
• interactive video generation
• low-bit DiT inference