Lost in Translation: How Does Bilingualism Shape Reader Preferences for Annotated Charts?

2025-03-19

AI Summary

This study investigates how bilingual background influences multilingual readers’ preferences for chart labeling density and semantic hierarchy, and their comprehension accuracy. Drawing on a large-scale experiment with 1,096 English–Tamil/Arabic bilinguals, it systematically evaluates six chart types across labeling density and semantic granularity dimensions. Using a cross-linguistic controlled design, mixed-effects modeling, cognitive-behavioral measures, and surveys, the study identifies, for the first time, a tripartite regulatory mechanism wherein language habit, intrapersonal code-switching frequency, and technical thinking disposition jointly modulate labeling preferences. Results show that full-text labeling significantly improves comprehension accuracy across all groups; English-dominant readers prefer high-label-density charts, whereas Tamil/Arabic-dominant readers favor either full-text or minimal-label variants; English-language labels are generally preferred—except among frequent code-switchers, who exhibit no such bias. The work proposes inclusive labeling design principles for multilingual users, addressing a critical gap in visualization human factors research concerning bilingual cognition.

Technology Category

• Knowledge Representation and Reasoning: Preferences
• Natural Language Processing: Machine Translation, Multilinguality, Cross-Lingual NLP
• Machine Learning: Learning Preferences or Rankings

Application Category

• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data

Abstract

Visualizations are powerful tools for conveying information but often rely on accompanying text for essential context and guidance. This study investigates the impact of annotation patterns on reader preferences and comprehension accuracy among multilingual populations, addressing a gap in visualization research. We conducted experiments with two groups fluent in English and either Tamil (n = 557) or Arabic (n = 539) across six visualization types, each varying in annotation volume and semantic content. Full-text annotations yielded the highest comprehension accuracy across all languages, while preferences diverged: English readers favored highly annotated charts, whereas Tamil/Arabic readers preferred full-text or minimally annotated versions. Semantic variations in annotations (L1-L4) did not significantly affect comprehension, demonstrating the robustness of text comprehension across languages. English annotations were generally preferred, with a tendency to think technically in English linked to greater aversion to non-English annotations, though this diminished among participants who regularly switched languages internally. Non-English annotations incorporating visual or external knowledge were less favored, particularly in titles. Our findings highlight cultural and educational factors influencing perceptions of visual information, underscoring the need for inclusive annotation practices for diverse linguistic audiences. All data and materials are available at: https://osf.io/ckdb4/.

Problem

• Impact of annotation patterns on multilingual reader preferences
• Effect of annotation volume and semantic content on comprehension
• Cultural and educational factors in visualization perception

Innovation

• Experiments with multilingual groups on visualization preferences
• Full-text annotations improve comprehension across all languages
• Cultural factors influence annotation preferences in visualizations