HierDoc: Hierarchical Page-to-Region Evidence Routing for Long-Document Visual Question Answering

2026-07-31

AI Summary

This work addresses the challenge of disjointed page-level and region-level evidence localization in multi-page document visual question answering, which hinders effective retrieval of sparse information. The authors propose a hierarchical evidence routing framework that models long-document evidence acquisition as a two-stage set prediction process: a page-level policy first identifies key pages, followed by a region-level policy that extracts semantic elements for the answer generation model. This approach uniquely decouples page and region selection into sequential, independently optimizable stages, enabling coordinated localization through granularity-specific structured set rewards. Integrating a two-tier policy architecture, staged GRPO optimization, fused OCR and table text representations, and joint global-local inputs, the method achieves state-of-the-art or competitive performance among open-source systems across multiple benchmarks, outperforming the strongest open-source baseline on LongDocURL by 16.87%, with region-level evidence boosting accuracy and F1 by 5.51% and 4.82%, respectively.

Abstract

Multi-page document visual question answering requires locating sparse evidence at both the page and region levels. Existing approaches typically emphasize one level over the other: page-centric methods focus on page acquisition, with region operations serving mainly as navigation aids, whereas region-centric methods assume that the relevant pages have already been supplied. Consequently, page and region selection remain disconnected rather than forming successive evidence decisions. We propose HierDoc, a hierarchical evidence-routing framework that formulates long-document evidence acquisition as two-stage set prediction from pages to regions. A page policy selects evidence pages from the full document; these pages are then parsed for semantic elements, after which a region policy selects the elements passed to a downstream answer model. Both answer-agnostic policies are optimized with stage-wise GRPO using granularity-specific structured-set rewards. The answer model receives selected full pages together with selected region crops and OCR or table text, preserving global context while emphasizing fine-grained evidence. Across the evaluated benchmarks, HierDoc achieves state-of-the-art or competitive performance among open-weight systems, improving LongDocURL by 16.87% relative to the strongest reported open-weight baseline. Controlled ablations further show that selected regional evidence improves the page-only system in accuracy and F1 by 5.51% and 4.82%, respectively. These results demonstrate the benefit of organizing coarse page routing and fine-grained region routing as successive, separately optimized stages of a unified evidence-acquisition process.

Problem

• visual question answering
• long-document
• evidence routing
• page-level selection
• region-level selection

Innovation

• hierarchical evidence routing
• two-stage set prediction
• page-to-region selection
• stage-wise GRPO
• long-document VQA