HelloWorld: Enabling Socially Interactive Characters in Video World Models

2026-08-05

AI Summary

Existing video world models struggle to support natural social interactions between users and virtual characters. To address this limitation, this work proposes an immersive video world model that enables key-triggered social responses from characters—such as waving or greeting—and integrates dynamic camera motion. Methodologically, the approach introduces a self-distillation pipeline using synthetic data to learn camera pose conditioning without additional annotations, and incorporates a training-free cross-attention mask modulation module to achieve precise temporal localization of interactive behaviors. Built upon a DiT architecture, the proposed generative model significantly outperforms baseline methods on the newly introduced HelloWorldBench benchmark, achieving state-of-the-art performance in social interaction quality, visual aesthetics, and camera-following fidelity.

Abstract

Despite the remarkable recent progress of video world models, social interaction between users and the characters within these worlds remains unsupported. To fill this gap, we present HelloWorld, a video world model that enables social interaction with in-world characters. With a single button press, users can prompt the on-screen character to respond toward the camera, e.g., turning to the viewer, waving, nodding, or speaking a short greeting. To make these interactions natural, we propose a self-distillation pipeline that finetunes the video generation model on data synthesized by itself. Each synthesized clip contains both social interactions and camera motion, allowing the model to learn camera-pose conditioning without degrading interaction quality. At inference, we further introduce a training-free module that determines when the interaction occurs. Upon a button press, it modulates the cross-attention masks of the DiT so that the interaction-related text prompt attends only to the frames within the press window, temporally localizing the character's response. We further build HelloWorldBench, a 400-sample benchmark with three social interaction metrics alongside three conventional metrics, for evaluation. Experiments demonstrate that HelloWorld surpasses a variety of baselines in interaction quality, while maintaining state-of-the-art picture aesthetics and camera-pose following. Project page: https://github.com/AlayaLab/HelloWorld

Problem

• social interaction
• video world models
• in-world characters
• camera-pose conditioning
• interactive characters

Innovation

• social interaction
• video world models
• self-distillation
• camera-pose conditioning
• cross-attention modulation