Political Alignment in Large Language Models: A Multidimensional Audit of Psychometric Identity and Behavioral Bias

2026-01-08

arXiv.org

AI Summary

This study addresses the critical gap in systematically evaluating the political orientations and behavioral biases of large language models (LLMs), which are increasingly deployed in societal decision-making yet pose risks to fairness and safety. The authors propose the first multidimensional political alignment auditing framework, integrating three established psychometric scales—Political Compass, SapplyValues, and 8 Values—and conduct an empirical analysis of 26 mainstream LLMs using a dataset of 27,000 annotated news items. Their findings reveal that 96.3% of models cluster in the libertarian-left quadrant, with closed-source models exhibiting pronounced cultural progressivism. A pervasive “center-left shift” bias is observed, alongside significantly higher accuracy in identifying far-left compared to far-right content. These results challenge the adequacy of unidimensional political assessments and raise concerns about the validity of the Political Compass in capturing cultural dimensions.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Humans and AI: Learning Human Values and Preferences

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Web Mining and Content Analysis: Large pretrained models with web data
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

As large language models (LLMs) are increasingly integrated into social decision-making, understanding their political positioning and alignment behavior is critical for safety and fairness. This study presents a sociotechnical audit of 26 prominent LLMs, triangulating their positions across three psychometric inventories (Political Compass, SapplyValues, 8 Values) and evaluating their performance on a large-scale news labeling task ($N \approx 27{,}000$). Our results reveal a strong clustering of models in the Libertarian-Left region of the ideological space, encompassing 96.3% of the cohort. Alignment signals appear to be consistent architectural traits rather than stochastic noise ($\eta^2>0.90$); however, we identify substantial discrepancies in measurement validity. In particular, the Political Compass exhibits a strong negative correlation with cultural progressivism ($r=-0.64$) when compared against multi-axial instruments, suggesting a conflation of social conservatism with authoritarianism in this context. We further observe a significant divergence between open-weights and closed-source models, with the latter displaying markedly higher cultural progressivism scores ($p<10^{-25}$). In downstream media analysis, models exhibit a systematic"center-shift,"frequently categorizing neutral articles as left-leaning, alongside an asymmetric detection capability in which"Far Left"content is identified with greater accuracy (19.2%) than"Far Right"content (2.0%). These findings suggest that single-axis evaluations are insufficient and that multidimensional auditing frameworks are necessary to characterize alignment behavior in deployed LLMs. Our code and data will be made public.

Problem

• Political Alignment
• Large Language Models
• Psychometric Bias
• Ideological Measurement
• Multidimensional Audit

Innovation

• multidimensional auditing
• political alignment
• large language models
• psychometric identity
• behavioral bias