🤖 AI Summary
This study systematically evaluates five representative VQA models—ABC-CNN, KICNLE, MVLM, BLIP-2, and OFA—addressing three core challenges: dataset bias, weak commonsense reasoning, and poor real-world generalization. Methodologically, it conducts the first unified, cross-paradigm, multi-dimensional analysis across bias robustness, cross-domain generalization, and implicit reasoning capability. The evaluation framework introduces novel, realistic-scenario-oriented assessment protocols. Key findings reveal substantial performance disparities: BLIP-2 achieves superior open-domain generalization, whereas OFA excels in fine-grained visual understanding; critically, all models attain <38% average accuracy on counterfactual questions, exposing a fundamental commonsense reasoning bottleneck. These results provide empirical grounding and methodological insights for future VQA model design and evaluation, emphasizing the need for bias-mitigated training, enhanced causal and counterfactual reasoning, and more ecologically valid benchmarks.
📝 Abstract
Visual Question Answering (VQA) has emerged as a pivotal task in the intersection of computer vision and natural language processing, requiring models to understand and reason about visual content in response to natural language questions. Analyzing VQA datasets is essential for developing robust models that can handle the complexities of multimodal reasoning. Several approaches have been developed to examine these datasets, each offering distinct perspectives on question diversity, answer distribution, and visual-textual correlations. Despite significant progress, existing VQA models face challenges related to dataset bias, limited model complexity, commonsense reasoning gaps, rigid evaluation methods, and generalization to real world scenarios. This paper presents a comprehensive comparative study of five advanced VQA models: ABC-CNN, KICNLE, Masked Vision and Language Modeling, BLIP-2, and OFA, each employing distinct methodologies to address these challenges.