IndicSQuAD: A Comprehensive Multilingual Question Answering Dataset for Indic Languages

2025-05-06

AI Summary

Indian languages are severely underrepresented in question answering (QA) systems, lacking dedicated multilingual QA resources. Method: We introduce IndicSQuAD—the first multilingual extractive QA dataset covering nine major Indian languages—systematically translated and aligned from SQuAD. We propose a multilingual collaborative answer span alignment mechanism, integrating cross-lingual boundary calibration and MahaSQuAD-enhanced translation to ensure high-fidelity translation and annotation consistency. Contribution/Results: We establish the first unified, hierarchical, and reproducible QA benchmark for Indian languages, releasing the full dataset (train/dev/test splits) alongside fine-tuned monolingual BERT and MuRIL-BERT models. Baseline experiments expose critical bottlenecks in low-resource language QA, particularly in domain transfer and precise answer span localization, thereby laying foundational infrastructure and empirical insights for QA research across Indo-Aryan and Dravidian languages.

Technology Category

• Natural Language Processing: Question Answering
• Machine Learning: Large Multimodal Models (LMMs)
• Knowledge Representation and Reasoning: Knowledge Representation Languages

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

The rapid progress in question-answering (QA) systems has predominantly benefited high-resource languages, leaving Indic languages largely underrepresented despite their vast native speaker base. In this paper, we present IndicSQuAD, a comprehensive multi-lingual extractive QA dataset covering nine major Indic languages, systematically derived from the SQuAD dataset. Building on previous work with MahaSQuAD for Marathi, our approach adapts and extends translation techniques to maintain high linguistic fidelity and accurate answer-span alignment across diverse languages. IndicSQuAD comprises extensive training, validation, and test sets for each language, providing a robust foundation for model development. We evaluate baseline performances using language-specific monolingual BERT models and the multilingual MuRIL-BERT. The results indicate some challenges inherent in low-resource settings. Moreover, our experiments suggest potential directions for future work, including expanding to additional languages, developing domain-specific datasets, and incorporating multimodal data. The dataset and models are publicly shared at https://github.com/l3cube-pune/indic-nlp

Problem

• Addressing underrepresentation of Indic languages in QA systems
• Creating multilingual QA dataset for nine Indic languages
• Evaluating performance challenges in low-resource language settings

Innovation

• Multilingual QA dataset for nine Indic languages
• Translation techniques for linguistic fidelity
• Monolingual and multilingual BERT model evaluation