BERT-based Models vs. Large Language Models for Low-Resource Named Entity Recognition: A Comparative Study on Marathi

2026-07-25

AI Summary

This study addresses the performance limitations of named entity recognition (NER) for Marathi, a low-resource language, which stem from scarce annotated data and linguistic complexity. The authors fine-tune the Marathi-specific language model MahaBERT-v2 on the MahaNER dataset and present the first systematic evaluation comparing its performance against general-purpose large language models, including Gemini, LLaMA-3.3-70B, and Gemma. Experimental results demonstrate that the fine-tuned MahaBERT-v2 achieves F1 scores of 0.88–0.91, outperforming the current MahaNER baseline (F1 = 0.8843) and substantially surpassing all evaluated general-purpose models, which attain F1 scores between 0.57 and 0.69. These findings underscore the critical importance of domain adaptation and language-specific modeling for effective NER in low-resource settings.

Abstract

Named Entity Recognition (NER) for low-resource languages such as Marathi remains a challenging task due to limited annotated resources and linguistic complexity. Although recent Large Language Models (LLMs) have demonstrated strong performance across a wide range of natural language processing tasks, their effectiveness for language-specific NER in low-resource settings remains uncertain. In this study, we fine-tune MahaBERT-v2 on different variants of the MahaNER dataset and systematically compare the performance of these models with an existing MahaNER baseline and prominent general-purpose LLMs, including Gemini, LLaMA-3.3-70B, and Gemma models. All models are evaluated on a Marathi NER test dataset using standard metrics of precision, recall, and F1-score. The experimental results show that the fine-tuned MahaBERT-based models consistently outperform both the baseline and all evaluated LLMs, with the fine-tuned models achieving F1-scores ranging from 0.88 to 0.91, surpassing the existing MahaNER model (0.8843) and significantly exceeding the performance of LLM-based approaches, whose F1-scores range from 0.57 to 0.69. These findings demonstrate that task-specific, language-focused models trained on domain-relevant data remain more effective than general-purpose LLMs for Marathi NER, highlighting the continued importance of specialized architectures for low-resource language processing.

Problem

• Named Entity Recognition
• Low-Resource Languages
• Marathi
• Large Language Models
• BERT-based Models

Innovation

• low-resource NER
• MahaBERT
• language-specific models
• large language models
• Marathi