Evaluating the Effectiveness of Persona Simulation in Opinion Prediction with GPT-4.1

2026-07-22

AI Summary

This study investigates the capacity of large language models to predict individuals’ opinions and behaviors on complex societal issues—such as political elections and public health—based on their demographic and psychographic profiles. We present the first systematic evaluation of GPT-4.1’s ability to simulate human personas for opinion forecasting across multiple domains, leveraging the Columbia University Personas dataset alongside real-world survey data from Pew Research Center’s American Trends Panel Wave 123. Through carefully engineered prompts, the model generates contextually grounded dialogues reflective of individual backgrounds to assess predictive validity. Results demonstrate that the model accurately forecasts election outcomes in eight out of nine U.S. states for the 2024 cycle and achieves a 0.94 accuracy in predicting attitudes toward childhood vaccination. While generated dialogues exhibit strong alignment with persona attributes, their naturalness and fluency remain areas for improvement.

Abstract

Persona simulation involves utilizing large language models (LLMs) to anticipate human choices or interactions based on specific characteristic information. To further understand current limitations and future directions, we tested persona simulation in opinion prediction with GPT-4.1 (knowledge cutoff by June 2024). Using personas from nine U.S. states provided by Columbia University's Personas dataset, GPT-4.1 accurately predicted 2024 election outcomes in eight out of the nine states, only failing in one of the swing states. We then focused on opinions related to medicine and healthcare. With the American Trends Panel Wave 123 dataset from Pew Research Center, GPT-4.1 was able to anticipate beliefs about childhood vaccines with an accuracy of up to 0.94. Furthermore, we applied GPT-4.1 to generate conversations among personas and observed that the simulated dialogues and opinions adhered well to personas' personalities and backgrounds, albeit lacking natural human-like flow. Persona simulation proves to be a promising application of artificial intelligence as long as biases are addressed. In the near future, it will be beneficial to apply it to opinion analysis and reaction prediction in diverse fields ranging from public health to lawmaking to economics.

Problem

• persona simulation
• opinion prediction
• large language models
• GPT-4.1
• human belief modeling

Innovation

• persona simulation
• opinion prediction
• large language models
• GPT-4.1
• human-like dialogue generation