BanglaForge: LLM Collaboration with Self-Refinement for Bangla Code Generation

2025-12-22

AI Summary

To address data scarcity and tooling limitations for low-resource Bangla in code generation, this paper proposes Coder-Reviewer—the first retrieval-augmented, dual-model collaborative framework. It integrates in-context learning, LLM-assisted translation, systematic prompt engineering, and an execution-feedback-driven multi-round self-refinement mechanism. The framework jointly optimizes natural language understanding and code robustness through encoder-reviewer co-modeling, with iterative corrections guided by program execution feedback. Evaluated on the newly constructed BLP-2025 benchmark, our approach achieves 84.00% Pass@1 accuracy, substantially outperforming existing baselines. This work introduces, for the first time, retrieval augmentation and execution-aware self-refinement to low-resource NL2Code tasks, establishing a scalable paradigm for code generation in resource-constrained languages.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Bangla is a low-resource language for code generation, lacking large-scale annotated datasets and tools to transform natural language specifications into executable programs. This makes Bangla-to-code generation a challenging task requiring innovative solutions. To address this, we introduce BanglaForge, a novel framework for generating code from Bangla function descriptions. BanglaForge leverages a retrieval-augmented dual-model collaboration paradigm with self-refinement, combining in-context learning, llm-based translation, systematic prompt engineering, and iterative self-refinement based on execution feedback, where a coder generates initial solutions and a reviewer enhances them for robustness. On the BLP-2025 Bangla Code Generation benchmark, BanglaForge achieves a competitive Pass@1 accuracy of 84.00%, demonstrating the effectiveness of retrieval, model collaboration, and self-refinement for low-resource Bangla code generation.

Problem

• Generates code from Bangla natural language descriptions
• Addresses low-resource challenges in Bangla code generation
• Uses collaborative LLMs with self-refinement for robustness

Innovation

• Retrieval-augmented dual-model collaboration paradigm
• Self-refinement with execution feedback iteration
• In-context learning and systematic prompt engineering