Evidence Attribution in Visual Document Understanding without Coordinates or Region Labels

2026-07-27

AI Summary

This work addresses the limitations of existing visual document understanding models, which rely on coordinate-based interfaces for evidence attribution and are prone to hallucination and inaccurate localization of supporting text regions. To overcome these issues, the authors propose a quotation-retrieval attribution mechanism that eliminates the need for explicit coordinates or region labels. The approach replaces coordinate outputs with natural language quotations and integrates a layout-aware multimodal retriever to locate evidence, coupled with an unsupervised training strategy based on GRPO. Experiments across six open-source vision-language models demonstrate substantial improvements: evidence recall increases by 26–47 percentage points, attribution hallucination rates drop by approximately 50%, and under fully unsupervised conditions—without any region-level supervision—the strict attribution accuracy of an 8B-parameter model rises from 22.4% to 33.8%, significantly outperforming conventional coordinate-based interfaces.

Abstract

Reliable visual document understanding requires a model to attribute each answer to the evidence regions that support it. Recent benchmarks and systems express this step through a coordinate interface: the model outputs the coordinates of bounding boxes that mark the evidence regions in the document. Under this interface, vision-language models often fail to identify the right regions even when the answer is correct, a failure known as Attribution Hallucination. We present a study that investigates whether this failure is partially limited by what the model can express through coordinates. On a verified bilingual CiteVQA subset, we compare the coordinate interface with a language interface in which the model outputs only text, quoting its evidence verbatim, and a multimodal retriever returns the location of each quote as a page region proposed by a layout parser (tables and figures are quoted through their captions or notes); the comparison is repeated over six open vision-language models. Compared with the coordinate interface, evidence recall rises from at most 8 points to between 26 and 47 and the hallucination rate roughly halves, with little change in answer quality. Building on this comparison, we use the same quote-and-retrieve pipeline as a training scaffold: because region-level evidence labels are expensive to collect for long documents, we introduce a GRPO recipe whose reward is a judge's reading of the gold answer and crops of the retrieved regions, training the model to quote better evidence without any region labels and raising an 8B backbone's strict attributed accuracy from 22.4 to 33.8. These findings indicate a practical path to improve attribution"without a coordinate interface and without costly region-level supervision.

Problem

• Evidence Attribution
• Visual Document Understanding
• Attribution Hallucination
• Coordinate Interface
• Region Labels

Innovation

• evidence attribution
• visual document understanding
• language interface
• attribution hallucination
• GRPO