The Language You Ask In: Language-Conditioned Ideological Divergence in LLM Analysis of Contested Political Documents

2026-01-17

AI Summary

This study reveals that large language models (LLMs) exhibit systematic ideological bias in multilingual political text analysis depending on the language of the prompt. By inputting semantically equivalent prompts in Russian and Ukrainian to the same LLM and analyzing its interpretations of identical Ukrainian civil society documents, the research—combining qualitative discourse analysis with comparative political text methods—provides the first empirical evidence that merely switching the prompt language can yield starkly opposing political stances. Specifically, Russian-language prompts tend to align with official Russian narratives and delegitimize civil society, whereas Ukrainian-language prompts resonate with Western liberal-democratic discourses and affirm its legitimacy. These findings underscore the critical role of prompt language in shaping LLMs’ ideological outputs and offer essential insights for mitigating bias in multilingual AI systems.

Technology Category

• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Learning Human Values and Preferences

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Large language models (LLMs) are increasingly deployed as analytical tools across multilingual contexts, yet their outputs may carry systematic biases conditioned by the language of the prompt. This study presents an experimental comparison of LLM-generated political analyses of a Ukrainian civil society document, using semantically equivalent prompts in Russian and Ukrainian. Despite identical source material and parallel query structures, the resulting analyses varied substantially in rhetorical positioning, ideological orientation, and interpretive conclusions. The Russian-language output echoed narratives common in Russian state discourse, characterizing civil society actors as illegitimate elites undermining democratic mandates. The Ukrainian-language output adopted vocabulary characteristic of Western liberal-democratic political science, treating the same actors as legitimate stakeholders within democratic contestation. These findings demonstrate that prompt language alone can produce systematically different ideological orientations from identical models analyzing identical content, with significant implications for AI deployment in polarized information environments, cross-lingual research applications, and the governance of AI systems in multilingual societies.

Problem

• language-conditioned bias
• ideological divergence
• large language models
• political document analysis
• multilingual AI

Innovation

• language-conditioned bias
• ideological divergence
• large language models
• cross-lingual analysis
• prompt engineering