Language Unalignability: Why Some Concepts Resist Cross-Cultural Benchmark Evaluation

2026-10-06

AI Summary

This study addresses the failure of the translation isomorphism assumption underlying multilingual large language model evaluation, which prevents lossless cross-lingual mapping of cultural concepts. Grounded in a cloud framework, this work formally defines α-unalignability and employs contextual embedding point-set modeling, neuron intervention, cycle consistency detection, and manifold curvature analysis to reveal the intrinsic unmapability of specific concepts within semantic structures across behavioral, mechanistic, and diagnostic dimensions, identifying a representation-intervention gap (RIG). Challenging single-scalar evaluation paradigms, the research constructs multidimensional diagnostic profiles demonstrating fundamental limitations in existing methods. Ultimately, this work provides a theoretical foundation for developing AI systems that respect cultural diversity.

Abstract

Current evaluation of multilingual Large Language Models (LLMs) rests on an implicit Translation-Isomorphism Assumption (TIA): that semantic structures across languages are congruent and mutually mappable without loss of information. We argue that this assumption is not merely violated in practice, but ill-posed in principle for a typologically identifiable class of concepts, including pragmatic markers, honorifics, and diachronically stratified terms. We formalize this failure using a usage-cloud framework, representing concepts as point sets of contextualized embeddings. We define $α$-unalignability as the impossibility of any mapping that simultaneously preserves lexical faithfulness (centroid correspondence) and structural faithfulness (local neighborhood topology). We provide three layers of evidence. Behaviorally, we show that FLORES-200 translation failures are predicted by language family and resource class but not by script, and that LOBSTER reasoning scores vary by family. Mechanistically, we report a Representation-Intervention Gap (RIG) in a nine-model case study on Yami: the models' activations encode a regularity along which Yami groups with other low-resource and Austronesian languages, yet interventions on language-specific neurons show no demonstrated advantage over random masks: the regularity is visible but not usable by this intervention. Finally, we operationalize these findings into a multidimensional diagnostic profile: Cycle-Consistency, Pragmatic-Load Disagreement, Manifold-Curvature Mismatch, and RIG. We argue that collapsing cultural competence into a single scalar incentivizes "probabilistic flattening," and that recognizing the unalignable class is a precondition for AI that respects, rather than erases, cultural divergence. This suggests that multilingual alignment is not a single well-defined objective, but a set of mutually incompatible projections.

Problem

• Multilingual LLM Evaluation
• Cross-Cultural Alignment
• Translation-Isomorphism Assumption
• Language Unalignability
• Semantic Structure

Innovation

• Language Unalignability
• Translation-Isomorphism Assumption
• Usage-Cloud Framework
• Representation-Intervention Gap
• Multidimensional Diagnostic Profile