Visual Graph Scaffolds for Structural Reasoning in Large Language Models

2026-06-01

AI Summary

Existing approaches treat graphs merely as external knowledge sources fed into large language models, overlooking their structural role in organizing reasoning processes. This work proposes, for the first time, leveraging graphs as internal reasoning scaffolds by employing schematic mind maps to guide models through multi-hop structured reasoning. The method integrates supervised fine-tuning with KL divergence-based knowledge distillation to train a student model to construct reasoning trajectories aligned with visual graph structures. Experimental results demonstrate that, even when direct answer cues are removed, this graph-guided strategy substantially outperforms flattened textual graph representations, enhancing answer quality without compromising reasoning efficiency. These findings validate the efficacy of graphs as intrinsic tools for structuring and guiding complex reasoning in language models.

Abstract

Graphs have been used to enhance large language models (LLMs) for structured reasoning, mostly as external knowledge sources are provided to models at test time. In this paper, we take a different view: the value of graphs for LLMs lie not only in supplying information, but also in organizing reasoning. Inspired by how humans use graph-structured mind maps to organize branching and converging thoughts, we ask whether graphs can serve as an internal form of reasoning assistance. We study this question on multi-hop question answering tasks, where teacher-provided reasoning traces are rewritten as graph mind maps and used to guide a student model. Our experiments reveal a clear modality gap. When graph structures are flattened into text, their benefits become limited once direct answer hints are removed. Under this abstract guidance setting, both reasoning efficiency and answer quality degrade substantially. In contrast, visual graph guidance remains effective without direct answer clues, and its advantage persists after supervised fine-tuning and KL-based distillation. The above findings support the claim that graphs should be studied not only as external knowledge structures for LLMs, but also as visual scaffolds for organizing reasoning.

Problem

• graph scaffolds
• structural reasoning
• large language models
• visual reasoning
• multi-hop question answering

Innovation

• visual graph scaffolds
• structured reasoning
• graph mind maps
• modality gap
• reasoning organization