Can Large Language Models Predict Associations Among Human Attitudes?

2025-03-26

AI Summary

This study investigates whether large language models (LLMs) can infer latent associations among human attitudes across disparate, heterogeneous domains—despite the absence of surface-level similarity—thereby probing their capacity for social reasoning about the deep structure of human belief systems. Method: Leveraging GPT-4o, we construct an original multi-topic attitude-response dataset and propose a zero-shot attitude correlation modeling framework that requires no fine-tuning or explicit prompting. Contribution/Results: Experiments demonstrate that GPT-4o accurately reconstructs pairwise attitude correlations at the individual level (r = 0.72) and significantly outperforms conventional baselines in cross-domain attitude prediction (+28.6% accuracy). To our knowledge, this is the first empirical evidence that LLMs implicitly encode and leverage structural regularities underlying human social cognition. These findings advance our understanding of the boundaries of social intelligence in foundation models and open new avenues for modeling higher-order human belief systems computationally.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Natural Language Processing: (Large) Language ModelsCognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactionsUser Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendationSocial Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Prior work has shown that large language models (LLMs) can predict human attitudes based on other attitudes, but this work has largely focused on predictions from highly similar and interrelated attitudes. In contrast, human attitudes are often strongly associated even across disparate and dissimilar topics. Using a novel dataset of human responses toward diverse attitude statements, we found that a frontier language model (GPT-4o) was able to recreate the pairwise correlations among individual attitudes and to predict individuals' attitudes from one another. Crucially, in an advance over prior work, we tested GPT-4o's ability to predict in the absence of surface-similarity between attitudes, finding that while surface similarity improves prediction accuracy, the model was still highly-capable of generating meaningful social inferences between dissimilar attitudes. Altogether, our findings indicate that LLMs capture crucial aspects of the deeper, latent structure of human belief systems.

Problem

• Can LLMs predict human attitudes across diverse topics?
• Does GPT-4o infer attitudes without surface-similarity cues?
• Do LLMs capture latent structures of human belief systems?

Innovation

• GPT-4o predicts human attitude correlations
• Model infers attitudes without surface similarity
• LLMs capture latent human belief structure