Rotated, but How Far? Diagnosing and Improving Object-Rotation Reasoning in VLMs

2026-10-05

AI Summary

This study addresses the inability of vision-language models (VLMs) to accurately estimate object rotation angles due to standard angle collapse. We reveal that VLMs inherently encode rotational information but fail to fully exploit it. To investigate this, we construct the OR-Bench benchmark and propose RotationCue, a lightweight decoder that leverages linear probing analysis and intermediate textual context injection to extract latent rotational features from frozen representations and feed them back into the model for enhanced reasoning. Experimental results demonstrate that our method improves macro-average accuracy by 7.9 to 12.6 percentage points across three VLMs while preserving their general-purpose capabilities.

Abstract

Vision-language models (VLMs) can detect that an object has rotated across views, but cannot reliably tell by how much. We introduce OR-Bench, a fine-grained benchmark for object-rotation reasoning with eight tasks covering rotation detection, rotation magnitude estimation, and multi-view rotation reasoning. Across 12 VLMs, the gap is stark: the strongest models approach 100% accuracy on detection, yet even coarse magnitude estimation is near chance. When asked for exact angles, models place 91.8--100% of their predictions on just $0^\circ$, $90^\circ$, and $180^\circ$, a failure we term canonical-angle collapse. This collapse persists even without visual input. Representation probing shows that missing information is only part of the explanation. Although rotation information becomes less recoverable at finer granularity, substantial coarse-grained information remains, and a simple linear probe outperforms the models'generated answers. This suggests that VLMs underuse rotation information they already encode. We therefore propose RotationCue, a lightweight decoder that recovers coarse rotation information from the VLM's own frozen representations and feeds it back to the model as intermediate textual context. Across three VLMs, RotationCue improves every model--task combination on OR-Bench, raising macro-average accuracy by 7.9--12.6 points while preserving general capabilities.

Problem

• Vision-Language Models
• Object-Rotation Reasoning
• Rotation Magnitude Estimation
• Canonical-Angle Collapse

Innovation

• Object-Rotation Reasoning
• Vision-Language Models
• Canonical-Angle Collapse
• Representation Probing
• RotationCue