ReGround: Restoring Visual Grounding in Multi-Step Reasoning through Self-Diagnosis and Visual Re-Examination

2026-08-04

AI Summary

This work addresses the tendency of vision-language models to over-rely on linguistic priors during multi-step reasoning, which degrades their visual grounding capabilities. The authors propose ReGround, a novel framework that integrates self-diagnosis with visual re-examination mechanisms without modifying model architecture or requiring external tools. By guiding capability development during data construction, ReGround enables models to autonomously detect grounding failures at inference time and selectively reinject relevant visual information. The approach employs a capability bootstrapping strategy, leveraging stronger models from the same family to provide diagnostic supervision for training a lightweight policy model that efficiently corrects errors. Evaluated across eight benchmarks and two backbone architectures, ReGround consistently improves performance—particularly on visually intensive multi-step reasoning tasks—while incurring significantly lower inference overhead than tool-augmented alternatives.

Abstract

Vision-Language Models (VLMs) often lose visual grounding during multi-step reasoning: as reasoning chains grow longer, later inference steps rely increasingly on language priors rather than image evidence. We identify a consistent benchmark-level signature associated with this degradation: across 2,510 re-examined samples from four benchmarks, attention entropy over image tokens typically decreases during Round 1 and rises again after image re-injection. However, we find that effective visual re-examination requires two complementary ingredients: image re-injection and targeted self-diagnosis. Without targeted diagnosis, re-examination can even hurt performance, whereas accurate self-diagnosis yields substantial gains -- a swing of several points on key benchmarks, indicating that diagnostic quality is a key factor in whether re-examination helps or hurts in our setting. We present ReGround, a two-stage framework that teaches VLMs to self-diagnose grounding failures and selectively re-examine visual evidence, without architectural modifications or external tools. Through capability bootstrapping, a stronger variant from the same model family provides diagnostic scaffolding only during data construction, while the policy model learns to diagnose autonomously at inference time and retains most of the assisted gains. Experiments on eight benchmarks across two VLM backbones demonstrate consistent gains, especially on visually intensive multi-step reasoning tasks, while incurring only modest inference overhead relative to tool-augmented baselines. Project page: https://sespoir.github.io/reground-page/ . Code: https://github.com/sespoir/ReGround .

Problem

• visual grounding
• multi-step reasoning
• vision-language models
• language priors
• image evidence

Innovation

• visual grounding
• self-diagnosis
• image re-examination
• vision-language models
• multi-step reasoning