Oral to Web: Digitizing'Zero Resource'Languages of Bangladesh

2026-03-05

AI Summary

This study addresses the critical gap in digital resources for approximately 40 Indigenous minority languages of Bangladesh, 14 of which are endangered and predominantly oral with no established writing systems. Through a 90-day field campaign, the project pioneered the simultaneous collection and standardized annotation of 42 language varieties spanning four major language families and unclassified languages. It establishes Bangladesh’s first national multimodal parallel corpus, comprising 85,792 structured text entries and 107 hours of audio recordings transcribed in the International Phonetic Alphabet (IPA), contributed by 77 native speakers. Hosted on the Multilingual Cloud platform, the corpus is publicly accessible and searchable, providing essential infrastructure for systematic digital archiving of zero-resource languages and advancing low-resource natural language processing research.

Technology Category

• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

We present the Multilingual Cloud Corpus, the first national-scale, parallel, multimodal linguistic dataset of Bangladesh's ethnic and indigenous languages. Despite being home to approximately 40 minority languages spanning four language families, Bangladesh has lacked a systematic, cross-family digital corpus for these predominantly oral, computationally"zero resource"varieties, 14 of which are classified as endangered. Our corpus comprises 85792 structured textual entries, each containing a Bengali stimulus text, an English translation, and an IPA transcription, together with approximately 107 hours of transcribed audio recordings, covering 42 language varieties from the Tibeto-Burman, Indo-European, Austro-Asiatic, and Dravidian families, plus two genetically unclassified languages. The data were collected through systematic fieldwork over 90 days across nine districts of Bangladesh, involving 16 data collectors, 77 speakers, and 43 validators, following a predefined elicitation template of 2224 unique items organized at three levels of linguistic granularity: isolated lexical items (475 words across 22 semantic domains), grammatical constructions (887 sentences across 21 categories including verbal conjugation paradigms), and directed speech (862 prompts across 46 conversational scenarios). Post-field processing included IPA transcription by 10 linguists with independent adjudication by 6 reviewers. The complete dataset is publicly accessible through the Multilingual Cloud platform (multiling.cloud), providing searchable access to annotated audio and textual data for all documented varieties. We describe the corpus design, fieldwork methodology, dataset structure, and per-language coverage, and discuss implications for endangered language documentation, low-resource NLP, and digital preservation in linguistically diverse developing countries.

Problem

• zero-resource languages
• endangered languages
• digital corpus
• language documentation
• low-resource NLP

Innovation

• zero-resource languages
• multimodal corpus
• endangered language documentation
• low-resource NLP
• IPA transcription