GraphVerse: A Comprehensive Visual Graph Reasoning Benchmark for Multimodal Large Language Models

2026-08-06

AI Summary

Current evaluation frameworks struggle to comprehensively assess multimodal large language models’ joint capabilities in perceiving structured visual information, understanding graph-like structures, and performing multi-step reasoning. To address this gap, this work proposes GraphVerse, a novel benchmark that introduces a graph-centric image editing (GIE) strategy to generate semantically consistent visual perturbations. It further designs VGR-Score, a process-sensitive automatic scoring mechanism, enabling unified evaluation of visual graph reasoning under both single-image and dual-image settings. Experimental results reveal critical limitations of existing models, demonstrate the effectiveness of GIE in preserving semantic integrity while inducing structural variations, and highlight GraphVerse’s transferable value across a broader range of multimodal reasoning tasks.

Abstract

Recent Multimodal Large Language Models (MLLMs) have achieved remarkable progress across diverse vision-language tasks, creating an urgent need for more challenging benchmarks. Yet existing evaluations still provide limited insight into whether these models can truly reason over structured visual information. Visual Graph Reasoning (VGR) offers a compelling testbed for this challenge, requiring models to integrate perception, structural understanding, and multi-step reasoning over graph-based visual inputs. However, prior VGR benchmarks often reduce the task to visual perception followed by text-based reasoning, restrict evaluation to single-image settings, rely on answer-only metrics, and underrepresent realistic graph-centric scenarios. To bridge the gap, we introduce GraphVerse, a unified benchmark that jointly evaluates perception, visual reasoning, and text-based graph reasoning in MLLMs under both single-image and paired-image settings. At its core is a suite of Graph-centric Image Editing (GIE) strategies that modify graph images while preserving their semantics, turning them into active tests of visual reasoning. We further propose VGR-Score, a process-sensitive metric that evaluates reasoning quality beyond final-answer accuracy. Extensive experiments reveal several key limitations of current MLLMs in VGR, while also validating the effectiveness of GIE strategies and the transferability of GraphVerse to broader multimodal reasoning capabilities. The code is available at https://github.com/sunyuanfu/GraphVerse.

Problem

• Visual Graph Reasoning
• Multimodal Large Language Models
• Benchmark
• Graph-centric Reasoning
• Reasoning Evaluation

Innovation

• Graph-centric Image Editing
• Visual Graph Reasoning
• Multimodal Large Language Models
• VGR-Score
• Paired-image Reasoning