Semantic Grading of Written Answers in Low-Resource Language Bangla Using a Fine-Tuned Lightweight Language Model

2026-06-10

AI Summary

This study addresses the challenge of limited effective automated scoring systems for low-resource languages like Bengali by proposing a bilingual (Bengali–English) approach that prioritizes semantic correctness over lexical overlap. The method leverages a synthetically generated bilingual dataset to efficiently fine-tune a lightweight Qwen3-8B model using QLoRA, jointly incorporating the question, reference answer, and student response to produce both numerical scores and context-aware, concise feedback. Designed for deployment efficiency and robustness against information leakage, the system demonstrates strong alignment with human raters in manual evaluation (Spearman ρ = 0.936, MAE = 0.725) and achieves state-of-the-art leakage resistance on synthetic benchmarks (RoRa = 0.819).

Abstract

Bangla is among the world's most widely spoken languages, yet it remains underserved in educational NLP research. In many remote and rural regions, access to qualified subject teachers is limited, and written answers are consequently graded largely by hand, restricting timely and consistent feedback. Automatic assessment is challenging because semantically correct responses can vary substantially in surface form. We present a bilingual (Bangla-English) evaluation system designed for low-resource educational settings that prioritizes semantic correctness over lexical overlap. Our approach fine-tunes a lightweight language model to grade each response using the question, reference answer, and student answer, producing a numeric score and concise, context-grounded feedback suitable for classroom deployment. We also construct a synthetic bilingual dataset to enable controlled training and evaluation. Across proprietary and open-source LLMs evaluated under a unified protocol, our QLoRA-tuned Qwen3-8B confirms consistent improvement by producing the most leakage-resistant feedback (RoRa = 0.819) in synthetic evaluation and the strongest agreement with human scores (rho = 0.936, MAE = 0.725) in a dedicated human study.

Problem

• semantic grading
• low-resource language
• Bangla
• automatic assessment
• written answer evaluation

Innovation

• semantic grading
• low-resource language
• lightweight language model
• QLoRA fine-tuning
• bilingual educational NLP