SEER: A Self-Grounded Evidence Interface for Controlled Spatial Relation Classification

2026-08-04

AI Summary

This work addresses the challenge that vision-language models (VLMs) often misclassify spatial relationships due to ambiguous entity localization or interference from global contextual cues. To mitigate this, the authors propose SEER, a training-free, inference-time evidence interface that enhances the reasoning of frozen VLMs by occluding candidate relations, constructing role-explicit query views, and integrating geometric information from both full-image and sparse bounding-box representations. SEER is the first method to explicitly model query-specific entity roles during inference and introduces an inverse-relation consistency check to refine predictions. Evaluated on GQA-Train900 and EmbSpatial, SEER improves accuracy by 3.94–11.79 percentage points across three frozen VLMs, demonstrating the effectiveness of localized focus and explicit role modeling in spatial reasoning.

Abstract

Spatial relation questions require a model to identify the queried subject and object before comparing their layout. Yet a VLM can recognize both entities and still answer from the wrong instance or an ambiguous global view. We ask whether making query-specific evidence explicit can mitigate this failure and propose SEER (Self-grounded Evidence for Entity-Relation Reasoning), a training-free inference-time evidence interface for frozen VLMs. SEER hides candidate relations during pair localization, constructs a query-specific view with explicit subject/object roles, and retains the full image and sparse box geometry as complementary evidence. For relation-choice protocols with exact inverse support, an optional refinement swaps the entity roles and changes the forward decision only when exactly one visual state obeys the corresponding inverse relation. On an image-disjoint GQA-Train900 test frozen before model scoring, SEER pools to +3.94 [2.17,5.72] over Full; the gain remains positive under label-independent grounding-order counterbalancing and on the 535 rows whose entity names are unique. The unchanged protocol yields +4.35 to +11.79 on all 2,434 filtered EmbSpatial pair-relation questions across three models. Matched controls separate local refocus from role-explicit conditioning. These results establish query-specific evidence construction as the principal intervention, with reciprocal consistency as a smaller protocol-specific refinement.

Problem

• spatial relation classification
• visual language models
• entity grounding
• relation reasoning
• evidence construction

Innovation

• self-grounded evidence
• spatial relation classification
• frozen VLMs
• role-explicit conditioning
• reciprocal consistency