PALMs: Using Multi Construct-Grounded Rationales for Modeling Population Preferences in LLMs

2026-08-02

AI Summary

This work addresses the challenge of accurately modeling systematic differences in values, beliefs, and cultural norms across diverse populations within large language models to faithfully simulate their preferences. The authors propose PALMs, a culturally grounded language model tailored to populations in the United States, India, Brazil, France, and Italy, which introduces psychology- and culture-informed reasoning rationales as implicit supervision signals for preference tuning. This approach requires no task-specific annotations and leverages construct alignment to model cross-cultural psychological dimensions. Evaluated across four key dimensions—personality, values, cultural norms, and morality—the method achieves an average relative improvement of 8.59%. It further demonstrates enhanced expressiveness, generalization, and transferability in personalized reward modeling (+5.19%), population simulation (+6.34%), and social reasoning tasks.

Abstract

Large language models are being extensively used to simulate individual user behavior, yet faithfully representing a population requires capturing the systematic variation in values, beliefs, and cultural norms that distinguish one group from another. We introduce Population Aligned Language Models (PALMs), a suite of models each aligned to specific populations, covering five countries: USA, India, Brazil, France and Italy. PALMs are created by synthesizing rationales grounded in psychological and cultural constructs and using these as latent supervision during preference tuning for population-specific alignment. Evaluated across four dimensions: personality, values and beliefs, cultural norms, and morality, PALMs consistently outperform baselines, including culture-specialized models, achieving an average of 8.59% relative improvement over the best baseline across all five populations. Notably, construct-grounded rationales outperform both demographic prompting and survey-based fine-tuning, suggesting that grounding preference learning in psychology and culture provides a richer inductive signal than surface-level response distributions. We further demonstrate strong generalization to downstream applications with- out task-specific supervision: outperforming best baselines by 5.19% in personalized reward modeling, 6.34% in population simulation, and showing strong transfer to social reasoning tasks. Datasets and code are available at: https://github.com/limenlp/PALMs.

Problem

• population preferences
• cultural norms
• values and beliefs
• large language models
• cross-cultural modeling

Innovation

• Population Alignment
• Construct-Grounded Rationales
• Preference Tuning
• Cultural Modeling
• Latent Supervision