BanglaMedQA and BanglaMMedBench: Evaluating Retrieval-Augmented Generation Strategies for Bangla Biomedical Question Answering

2025-11-06

AI Summary

To address knowledge-access inequity in low-resource language biomedical question answering (e.g., Bengali), this work introduces the first large-scale Bengali medical multiple-choice question benchmark dataset and proposes Proxy-RAG, a novel retrieval-augmented generation framework. Proxy-RAG synergistically integrates OCR-extracted local medical textbooks with web-based retrieval sources, enabling dynamic switching between retrieval and reasoning strategies to jointly optimize factual consistency and reasoning robustness. Extensive experiments on the OpenAI/gpt-oss-120b model demonstrate that Proxy-RAG achieves 89.54% accuracy—significantly outperforming conventional RAG, zero-shot fallback, iterative feedback, and ensemble-based RAG baselines. This study constitutes the first systematic empirical validation of RAG’s efficacy in low-resource biomedical QA. It provides the community with a reproducible benchmark dataset, an adaptable methodological framework, and rigorous empirical evidence—thereby advancing non-English medical AI research and deployment.

Technology Category

• Natural Language Processing: Question Answering
• Machine Learning: Calibration & Uncertainty Quantification
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Web Mining and Content Analysis: Community question answering

Abstract

Developing accurate biomedical Question Answering (QA) systems in low-resource languages remains a major challenge, limiting equitable access to reliable medical knowledge. This paper introduces BanglaMedQA and BanglaMMedBench, the first large-scale Bangla biomedical Multiple Choice Question (MCQ) datasets designed to evaluate reasoning and retrieval in medical artificial intelligence (AI). The study applies and benchmarks several Retrieval-Augmented Generation (RAG) strategies, including Traditional, Zero-Shot Fallback, Agentic, Iterative Feedback, and Aggregate RAG, combining textbook-based and web retrieval with generative reasoning to improve factual accuracy. A key novelty lies in integrating a Bangla medical textbook corpus through Optical Character Recognition (OCR) and implementing an Agentic RAG pipeline that dynamically selects between retrieval and reasoning strategies. Experimental results show that the Agentic RAG achieved the highest accuracy 89.54% with openai/gpt-oss-120b, outperforming other configurations and demonstrating superior rationale quality. These findings highlight the potential of RAG-based methods to enhance the reliability and accessibility of Bangla medical QA, establishing a foundation for future research in multilingual medical artificial intelligence.

Problem

• Developing accurate biomedical QA systems for low-resource Bangla language
• Evaluating retrieval-augmented generation strategies for medical reasoning
• Improving factual accuracy through textbook and web retrieval integration

Innovation

• Combining textbook and web retrieval with reasoning
• Implementing Agentic RAG for dynamic strategy selection
• Using OCR-processed Bangla medical textbook corpus