Be Consistent! Enhancing Robust Visual Reasoning in LVLMs with Consistency Constraints

2026-07-23

AI Summary

This work addresses the prevalent lack of logical consistency in existing large vision-language models (VLMs) when performing complex visual reasoning, a limitation exacerbated by the inadequacy of current benchmarks to comprehensively evaluate model robustness and reliability. To bridge this gap, the authors introduce ConVBench, the first benchmark specifically designed to assess logical consistency in complex visual reasoning, featuring six task categories with each image paired with two logically equivalent questions. They further propose ConVLM, a novel model trained via a Group Relative Policy Optimization (GRPO) reinforcement learning framework that jointly optimizes accuracy and consistency through a dual-reward mechanism, obviating the need for strict answer supervision. Experimental results demonstrate that this approach substantially improves both logical consistency and robust accuracy on ConVBench, validating its effectiveness under both strongly supervised and weakly supervised settings.

Abstract

While Large Vision-Language Models (LVLMs) exhibit strong perceptual capabilities, they remain vulnerable in visual reasoning tasks. Existing benchmarks largely focus on symbolic mathematical or scientific problems and simple vision-centric tasks, offering limited assessment of complex visual reasoning and logical consistency, a critical requirement for reliable reasoning systems. We introduce ConVBench, a complex vision-centric reasoning benchmark in which each image is paired with two logically equivalent questions across six categories: action and state, complex counting, spatial reasoning, causal and intent understanding, commonsense reasoning, and temporal perception. To complement this benchmark, we define two evaluation metrics, logical consistency and robust accuracy, that jointly assess both the correctness and consistency of model responses. We further present ConVLM, which improves LVLM reasoning through Group Relative Policy Optimization (GRPO)-based reinforcement learning with a novel consistency reward. This method leverages automatically generated logically equivalent question-answer pairs and a dual-reward design combining accuracy- and consistency-based signals, encouraging agreement between paired responses. The framework functions effectively with or without strict answer supervision.

Problem

• visual reasoning
• logical consistency
• Large Vision-Language Models
• robustness
• benchmarking

Innovation

• Consistency Constraints
• Large Vision-Language Models
• Logical Consistency
• Reinforcement Learning
• Visual Reasoning Benchmark