ATM: Action-Consistency Transfer Matrix for Diagnosing and Improving Latent World Models

2026-06-08

AI Summary

Current latent world models rely on time-consuming black-box simulation for planning evaluation, resulting in low efficiency. This work proposes the Action-consistency Transition Matrix (ATM) method, which employs a lightweight posterior probe to measure the semantic consistency of actions between real and predicted latent state transitions. The approach yields an interpretable diagnostic matrix that identifies representation deficiencies and provides training signals without requiring modifications to the planner. For the first time, action consistency is leveraged both as an interpretable diagnostic tool and as an optimization objective. ATM enables highly reliable model ranking on non-trivial tasks, accelerating CEM-based evaluation from minutes or hours to seconds—a speedup of over two orders of magnitude—while significantly improving downstream planning performance.

Abstract

Latent world models are increasingly used for control and goal-conditioned planning, yet assessing whether their learned representations are useful for planning usually requires slow, planner-coupled simulator evaluation with CEM or similar planners. Such evaluation is black-box and model-complexity-dependent: under the same protocol, different world models may require minutes to hours per checkpoint. In this work, we propose ATM, an Action-Consistency Transfer Matrix for diagnosing whether latent transitions preserve action semantics relevant to planning. ATM compares action information in real encoded transitions and model-predicted transitions through lightweight post-hoc probes, producing an interpretable matrix that reveals representation quality, transition-domain inconsistency, and failure modes without simulator rollout. It can also be collapsed into a simple screening score for within-task ranking across checkpoints, variants, and world models. When the true success gap is non-trivial, ATM achieves highly reliable pairwise ranking, while reducing minutes-to-hours CEM evaluation to seconds-level transition analysis, yielding more than 100x speedup in our setup. We further introduce AITS, showing that action-identifiability is not only diagnostic but also a useful training signal for improving downstream planning without changing the planner.

Problem

• latent world models
• planning
• representation evaluation
• action semantics
• model diagnosis

Innovation

• Action-Consistency Transfer Matrix
• Latent World Models
• Post-hoc Probing
• Action Identifiability
• Planning Evaluation