Few-Shot Prompting for Extractive Quranic QA with Instruction-Tuned LLMs

2025-08-08

AI Summary

This work addresses the challenges of extractive question answering (QA) over Quranic Arabic texts—characterized by linguistic complexity, domain-specific terminology, and deep semantic structure—under few-shot settings. Methodologically, it introduces a dedicated Arabic instruction-tuning prompt framework, augmented by a three-stage post-processing pipeline: subword alignment, overlap suppression, and semantic filtering, which collectively mitigate hallucination and improve answer span localization accuracy. Experiments employ instruction-tuned large language models (e.g., Gemini, DeepSeek) without task-specific fine-tuning, achieving significant gains over supervised fine-tuning baselines; the best configuration attains pAP10 = 0.637. To our knowledge, this is the first systematic validation of high-quality prompt engineering combined with domain-adapted post-processing for QA on low-resource, high-semantic-density religious texts. The proposed approach establishes a reusable technical paradigm for intelligent understanding of classical Arabic religious literature.

Technology Category

• Natural Language Processing: Question Answering
• Data Mining & Knowledge Management: Intelligent Query Processing
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Web question answering
• Web Mining and Content Analysis: Community question answering
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This paper presents two effective approaches for Extractive Question Answering (QA) on the Quran. It addresses challenges related to complex language, unique terminology, and deep meaning in the text. The second uses few-shot prompting with instruction-tuned large language models such as Gemini and DeepSeek. A specialized Arabic prompt framework is developed for span extraction. A strong post-processing system integrates subword alignment, overlap suppression, and semantic filtering. This improves precision and reduces hallucinations. Evaluations show that large language models with Arabic instructions outperform traditional fine-tuned models. The best configuration achieves a pAP10 score of 0.637. The results confirm that prompt-based instruction tuning is effective for low-resource, semantically rich QA tasks.

Problem

• Extractive QA on Quran with complex language
• Few-shot prompting for low-resource Arabic QA
• Improving precision in span extraction with post-processing

Innovation

• Few-shot prompting with instruction-tuned LLMs
• Specialized Arabic prompt framework for extraction
• Post-processing with alignment and semantic filtering