Bangla Sign Language Translation: Dataset Creation Challenges, Benchmarking and Prospects

2025-11-26

AI Summary

Sentence-level Bangladeshi Sign Language (BdSL) translation has long been hindered by severe scarcity of sentence-aligned, annotated data. Method: We introduce IsharaKhobor—the first standardized, publicly available sentence-level sign language translation dataset—comprising authentic news-domain multimodal videos, 3D skeletal keypoint sequences, corresponding Bengali text, and semantically normalized annotations. We propose a keypoint- and RQE-embedding-based baseline model and conduct systematic ablation studies incorporating lexical constraints and semantic normalization. Contribution/Results: Experiments demonstrate IsharaKhobor’s effectiveness, yielding substantial performance gains in low-resource BdSL translation. The dataset fills a critical gap in South Asian low-resource sign language resources, establishing a benchmark dataset, unified evaluation protocol, and reproducible modeling framework for BdSL translation—thereby advancing AI-assisted technologies for the Deaf and hard-of-hearing community.

Technology Category

• Computer Vision: Language and Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Multimodal Learning

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets

Abstract

Bangla Sign Language Translation (BdSLT) has been severely constrained so far as the language itself is very low resource. Standard sentence level dataset creation for BdSLT is of immense importance for developing AI based assistive tools for deaf and hard of hearing people of Bangla speaking community. In this paper, we present a dataset, IsharaKhobor , and two subset of it for enabling research. We also present the challenges towards developing the dataset and present some way forward by benchmarking with landmark based raw and RQE embedding. We do some ablation on vocabulary restriction and canonicalization of the same within the dataset, which resulted in two more datasets, IsharaKhobor_small and IsharaKhobor_canonical_small. The dataset is publicly available at: www.kaggle.com/datasets/hasanssl/isharakhobor [1].

Problem

• Creating standard sentence-level datasets for Bangla Sign Language Translation
• Developing AI assistive tools for deaf Bangla-speaking communities
• Addressing vocabulary restriction and canonicalization challenges in BdSLT

Innovation

• Created IsharaKhobor dataset for Bangla Sign Language
• Used landmark-based raw and RQE embedding benchmarking
• Applied vocabulary restriction and canonicalization techniques