The GUI Is Not the State: Diagnosing State Aliasing in GUI World Models

2026-09-26

AI Summary

This study addresses the state aliasing problem in GUI world models, where reliance solely on visible interfaces causes identical observations to correspond to divergent future states. We systematically formalize this issue for the first time and introduce StateAliasBench, a diagnostic benchmark for its evaluation. To resolve it, we propose a predictive state recovery method that reconstructs latent states through deterministic state interfaces and structured inference. Furthermore, we construct a unified estimator by integrating family-specific expert models with multi-teacher knowledge distillation, optimized via a frozen augmentation technique. This approach substantially improves state-sensitive predictions while preserving generative fidelity, ultimately enhancing the downstream task performance of GUI agents in AndroidWorld.

Abstract

GUI World Models (GUI-WMs) are increasingly used to predict future states for agent planning and simulation, yet most existing formulations condition only on the current GUI observation and action. We identify state aliasing, where the vis- ible interface omits transition-relevant environment state, so identical observable conditions can correspond to different valid futures. To diagnose this failure mode, we introduce StateAliasBench, a diagnostic benchmark that explicitly isolates such ambiguities via strict pairing. We further propose lightweight predictive- state recovery that infers structured state from history and augments otherwise frozen GUI-WMs through a deterministic state interface. Family-specific special- ists provide state recovery across heterogeneous state types, and multi-teacher dis- tillation consolidates them into a single unified estimator. Experiments show that existing GUI-WMs exhibit systematic failures under observation-only condition- ing, while predictive-state augmentation substantially restores state-sensitive pre- diction across evaluated WMs, preserves generative fidelity, and improves down- stream performance of GUI agents on AndroidWorld. These results suggest that reliable GUI world modeling should account not only for what is visible, but also for the hidden transition state that determines what happens next.

Problem

• GUI World Models
• State Aliasing
• Future State Prediction
• Hidden State

Innovation

• State Aliasing
• GUI World Models
• Predictive-State Recovery
• Multi-Teacher Distillation
• StateAliasBench