Readable Yet Unpredictable: Rotated-Outcome Prediction in Vision-Language Models

2026-06-01

AI Summary

Current vision-language models struggle to accurately infer the visual content of an image after a 180° rotation, revealing a deficiency in causal reasoning under spatial transformations. To address this limitation, this work introduces the Rotated-Outcome Prediction task—the first of its kind—and presents RotOutBench, a dedicated benchmark comprising paired image–text data, controlled rotation experiments, and analyses of model internal states. Systematic evaluation demonstrates that while state-of-the-art models excel at standard visual understanding tasks, their performance on rotated-outcome prediction remains near chance level. This stark discrepancy highlights a significant gap between their capabilities in perceptual recognition and generative reasoning about spatially transformed scenes.

Abstract

Can vision-language models predict what a 180° rotation would reveal from the original image alone? We study this ability through Rotated-Outcome Prediction: given an original image, a model must answer what would be seen or read after a 180° in-plane rotation, without directly observing the rotated target. To isolate this gap, we introduce RotOutBench, a paired diagnostic benchmark spanning open visual cases and controlled text-image rotations. A sharp pattern emerges: many VLMs can recognize the relevant content when directly given either the original or rotated image, yet fail to infer the rotated result from the original image alone. On controlled text-image rotations, predicted-rotation accuracy collapses to near zero even for models with high direct-reading accuracy. A model-level case study further shows that the prediction state can approach a rotated-image reading state, while the final readout still shifts toward the original string. Current VLMs can recognize a transformed visual state when it is shown, but often fail to predict that state from the original view.

Problem

• Rotated-Outcome Prediction
• Vision-Language Models
• 180-degree rotation
• visual reasoning
• image transformation

Innovation

• Rotated-Outcome Prediction
• Vision-Language Models
• Visual Reasoning
• RotOutBench
• Spatial Transformation