The Generator-Eraser Paradox: Community Guidelines for Responsible LLM-Assisted Dialect Resource Creation

2026-06-04

AI Summary

This study addresses the risk that large language models (LLMs) may inadvertently accelerate dialect attrition when deployed in dialect resource development, manifesting through a preference for prestige varieties, orthographic homogenization, and synthetic feedback loops—particularly endangering diglossic communities, those with weak written standards, or marginalized linguistic groups. Introducing the “generator-eraser paradox” as a novel theoretical framework, this work integrates variationist sociolinguistics and corpus linguistics with technical approaches such as retrieval-augmented generation and metadata annotation to formulate twelve community-oriented guidelines emphasizing dialect authenticity, variability, and community sovereignty. Validated through a case study on Arabic dialects, the proposed principles effectively address challenges related to diglossia, orthographic diversity, and community governance, offering a scalable paradigm for responsible LLM deployment in multilingual dialect contexts.

Abstract

Dialect resources occupy a unique position at the intersection of scientific description, cultural preservation, and computational infrastructure. Large language models offer powerful capabilities for accelerating dialect resource development through retrieval-grounded drafting, corpus navigation, metadata enrichment, and annotation workflow support. However, the same systems pose substantial risks: they can contribute to dialect erasure by privileging prestige varieties, homogenizing orthography, and enabling synthetic feedback loops that reduce linguistic diversity over time. These risks are particularly acute for language varieties characterized by diglossia, limited written standardization, or marginalized speaker communities. This paper makes three contributions. First, we integrate insights from variationist sociolinguistics and corpus linguistics to formalize the generator-eraser paradox as a theoretical framework for understanding the dual nature of LLM-assisted dialect work. Second, we derive 12 community guidelines that operationalize this framework into implementable design requirements for dialect resource creation and documentation. Third, we provide an in-depth case study of Arabic dialects, including a structured comparison of widely used resources, to demonstrate how these guidelines address language-specific challenges including diglossia, orthographic variability, and community governance. The contribution is conceptual and operational rather than experimental, with the goal of enabling dialect communities and resource builders across languages to adopt LLMs without sacrificing authenticity, variation, or sovereignty.

Problem

• dialect erasure
• large language models
• linguistic diversity
• diglossia
• language preservation

Innovation

• Generator-Eraser Paradox
• dialect resource creation
• large language models
• linguistic diversity
• community guidelines