NepaliGPT: A Generative Language Model for the Nepali Language

2025-06-19

AI Summary

Nepali lacks dedicated generative large language models (LLMs), severely constraining downstream task research. Method: We introduce NepaliGPT—the first open-source, autoregressive LLM specifically designed for Nepali. Our approach comprises (i) constructing a large-scale, domain-diverse Devanagari-script text corpus; (ii) designing the first Nepali question-answering benchmark (4,296 QA pairs); and (iii) developing a customized evaluation framework incorporating ROUGE scores, causal coherence, and causal consistency metrics. Results: NepaliGPT achieves a perplexity of 26.32 on text generation, ROUGE-1 of 0.2604, causal coherence of 81.25%, and causal consistency of 85.41%. This work fills a critical gap in foundational Nepali LLMs and establishes a reusable paradigm—including data, model architecture, and evaluation methodology—for low-resource language LLM development.

Technology Category

• Natural Language Processing: Generation
• Machine Learning: Large Multimodal Models (LMMs)
• Cognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

After the release of ChatGPT, Large Language Models (LLMs) have gained huge popularity in recent days and thousands of variants of LLMs have been released. However, there is no generative language model for the Nepali language, due to which other downstream tasks, including fine-tuning, have not been explored yet. To fill this research gap in the Nepali NLP space, this research proposes extit{NepaliGPT}, a generative large language model tailored specifically for the Nepali language. This research introduces an advanced corpus for the Nepali language collected from several sources, called the Devanagari Corpus. Likewise, the research introduces the first NepaliGPT benchmark dataset comprised of 4,296 question-answer pairs in the Nepali language. The proposed LLM NepaliGPT achieves the following metrics in text generation: Perplexity of 26.32245, ROUGE-1 score of 0.2604, causal coherence of 81.25%, and causal consistency of 85.41%.

Problem

• No generative language model exists for Nepali language
• Lack of benchmark datasets for Nepali NLP tasks
• Need for tailored LLM to improve Nepali text generation

Innovation

• Generative LLM tailored for Nepali language
• Introduces Devanagari Corpus for Nepali NLP
• First NepaliGPT benchmark dataset with QA pairs