Who Has The Final Say? Conformity Dynamics in ChatGPT's Selections

2025-10-30

AI Summary

This study investigates large language models’ (LLMs) susceptibility to social influence in high-stakes decision-making, focusing on conformity behavior in simulated hiring contexts. Through three pre-registered experiments, we examine how GPT-4o adjusts its judgments under social pressure—including unanimous group opposition and isolated dissent. Results demonstrate, for the first time, that LLMs exhibit both informational conformity (adopting others’ information) and normative conformity (aligning with perceived group consensus): conformity rates approach 100% under unanimous group opposition and remain as high as 40% even when only a single individual dissents. Concurrently, the model’s self-reported confidence significantly declines. These findings challenge the foundational assumption of LLMs as socially neutral decision aids, revealing intrinsic social dependency in their judgment processes. The work provides critical empirical evidence for AI ethics, human-AI collaborative decision-making, and robustness evaluation of foundation models.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Learning Human Values and Preferences
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Social Networks and Social Media: Generative AI / large language models and their impact on social systems

Abstract

Large language models (LLMs) such as ChatGPT are increasingly integrated into high-stakes decision-making, yet little is known about their susceptibility to social influence. We conducted three preregistered conformity experiments with GPT-4o in a hiring context. In a baseline study, GPT consistently favored the same candidate (Profile C), reported moderate expertise (M = 3.01) and high certainty (M = 3.89), and rarely changed its choice. In Study 1 (GPT + 8), GPT faced unanimous opposition from eight simulated partners and almost always conformed (99.9%), reporting lower certainty and significantly elevated self-reported informational and normative conformity (p < .001). In Study 2 (GPT + 1), GPT interacted with a single partner and still conformed in 40.2% of disagreement trials, reporting less certainty and more normative conformity. Across studies, results demonstrate that GPT does not act as an independent observer but adapts to perceived social consensus. These findings highlight risks of treating LLMs as neutral decision aids and underline the need to elicit AI judgments prior to exposing them to human opinions.

Problem

• Examining ChatGPT's susceptibility to social influence in decisions
• Investigating conformity dynamics when GPT faces unanimous opposition
• Highlighting risks of treating AI as neutral decision aids

Innovation

• GPT-4o tested for conformity in hiring decisions
• Model adapts to social consensus under opposition
• AI judgments should precede human opinion exposure