WorldExam: Benchmarking World Models from Apparent Appearance to Inherent Reactivity

2026-08-03

AI Summary

This work addresses a critical gap in the evaluation of current video generation models, which predominantly focus on visual quality and explicit instruction following while neglecting systematic assessment of intrinsic world reactivity. To this end, the authors propose WorldExam—the first hierarchical diagnostic benchmark that encompasses four evaluation tiers: visual quality, control adherence, spatial consistency, and world reactivity, with the latter introduced for the first time as a dedicated evaluation dimension. The benchmark comprises 1,474 cases across eight tasks and supports unified evaluation of camera-, action-, and language-driven models. Comprehensive evaluation of 20 representative models reveals significant performance disparities across the tiers, indicating that no existing method simultaneously achieves broad task coverage and strong consistency.

Abstract

Controllable video generation models are increasingly being developed as world models. Accordingly, evaluating them in this role extends beyond the apparent appearance of generated videos to the inherent reactivity of the worlds they depict: the ability to infer from the scene state how the world should react and to generate plausible consequences not explicitly described in the input. Yet existing benchmarks mainly assess visual quality or explicit instruction fulfillment by checking whether requested actions and interaction outcomes are realized, leaving inherent reactivity underexamined. We introduce WorldExam, a hierarchical diagnostic benchmark spanning four levels: Visual Quality, Control Adherence, Spatial Consistency, and World Reactivity. It comprises 1,474 cases across eight dedicated tasks and supports unified evaluation of camera-, action-, and language-driven model paradigms. The World Reactivity level evaluates scene-conditioned reactions and goal-directed behaviors beyond what is explicitly specified in the input. Evaluation of 20 representative models reveals a clear capability split. Camera-driven models excel at camera control, but their interfaces do not support dynamic interaction; action-driven models control subjects more precisely but often leave the world unresponsive; and language-driven models perform better on interaction but follow complex controls less faithfully. No model combines broad task coverage with consistently strong performance, showing that high visual quality and explicit instruction fulfillment do not guarantee inherent reactivity.

Problem

• world models
• inherent reactivity
• video generation
• benchmarking
• controllable generation

Innovation

• world models
• inherent reactivity
• controllable video generation
• diagnostic benchmark
• causal reasoning