DDVT: Dynamic Dual-level Vision Transformer Fusion Network for Answer Grounding in Visual Question Answering

2026-07-26

AI Summary

This study addresses the answer localization task in visual question answering, which aims to precisely identify image regions relevant to the answer based on a natural language question. To this end, the authors propose a dynamic dual-level vision Transformer fusion network that innovatively integrates a question-guided dynamic region (QGDR) module and a cross-modal multi-scale aggregation (CMA) module. These components enable fine-grained alignment and fusion between textual queries and visual features at both pixel and region levels. The proposed method significantly outperforms current state-of-the-art approaches across multiple established visual question answering benchmarks, demonstrating substantial improvements in answer localization accuracy.

Abstract

Answer grounding in visual question answering aims to locate the region from a given natural language question associated with the visual content of an image, which has garnered significant attention due to its practical applications. In this paper, we introduce the Dynamic Dual-level Vision Transformer Fusion Network (DDVT) for answer grounding in visual question answering. Specifically, we propose a question-guided dynamic regional-level module (QGDR) that combines complementary image context through ROI Align and text content, enabling precise localization of text-related visual content. Moreover, we present a cross-modal multi-scale aggregation module (CMA) that enhances feature fusion between pixel-level and region-level features, facilitating the effective localization of visual content associated with grounded answers. Furthermore, we fuse the located visual content with text features to locate the region and provide answers to questions posed about the image. Experimental results demonstrate that our DDVT outperforms state-of-the-art methods on several widely-used benchmarks.

Problem

• answer grounding
• visual question answering
• region localization
• vision-language alignment

Innovation

• Dynamic Dual-level Vision Transformer
• Question-guided Dynamic Regional-level
• Cross-modal Multi-scale Aggregation
• Answer Grounding
• Visual Question Answering