AlayaWorld: Interactive Long-Horizon World Modeling -- Full Technical Report

2026-07-20

AI Summary

This work addresses the limitations of traditional game development, which relies on manual pipelines and struggles to generate interactive, continuously evolving virtual worlds in real time. The authors propose an interactive long-horizon video world model based on a 15B-parameter video diffusion Transformer, capable of autoregressively generating 540p/720p videos at 24 fps conditioned on camera trajectories and switchable text prompts. Key innovations include a bounded visual context mechanism—integrating persistent anchor frames, compressed temporal history, geometry-aligned spatial memory, and recent-frame conditioning—and a discrete autoregressive distillation framework. Evaluated on iWorld-Bench, the method achieves state-of-the-art performance, reducing inference sampling steps per frame from approximately 30 to just 4, thereby significantly improving long-term consistency, inference efficiency, and generation stability.

Abstract

Unlike conventional video game development, which relies on labor-intensive pipelines for asset production, animation, physics, and programming, video world models generate interactive environments from user inputs instantly. It enable us to create customized, explorable, and continuously evolving virtual world from text, an image, or video. Realizing this vision requires four tightly coupled capabilities: interaction, persistent spatiotemporal consistency, stable long-horizon generation, and efficient response. We present AlayaWorld, an interactive long-horizon video world model that generates 24-fps video at 540p and 720p. Built on a 15B video diffusion transformer, AlayaWorld generates short latent chunks autoregressively under camera trajectories and switchable text prompts. Its bounded visual context combines a persistent sink frame, compressed temporal history, geometry-aligned spatial memory, and recent-frame conditioning. To reduce long-term drift, the model is trained with corrupted histories and prediction residuals collected from its own roll-outs. We further introduce a discrete autoregressive distillation formulation that combines distribution-matching distillation, self-forcing++, and consistency distillation, reducing inference from approximately 30 sampling steps to four steps per chunk. On iWorld-Bench, AlayaWorld achieves the best performance over long-horizon generation. Conceived as a full-stack, open-source, and long-term project, AlayaWorld is intended to provide an extensible foundation for future research on interactive video world models.

Problem

• interactive world modeling
• long-horizon generation
• spatiotemporal consistency
• video diffusion
• autoregressive generation

Innovation

• interactive world modeling
• long-horizon video generation
• video diffusion transformer
• spatiotemporal consistency
• autoregressive distillation