When Prompting Fails to Sway: Inertia in Moral and Value Judgments of Large Language Models

2024-08-16

AI Summary

This work identifies a “value inertia” phenomenon in large language models (LLMs) under persona-based prompting: despite diverse role assignments, models exhibit highly stable (>87%) unidirectional biases across core moral dimensions—particularly harm avoidance and fairness—contradicting the expected plasticity of moral stance. To investigate, we propose a novel paradigm integrating macro-trend analysis with large-scale persona enactment, grounded in Moral Foundations Theory (MFT) to quantify multi-dimensional value orientations. Statistical significance testing across mainstream LLMs confirms robust, cross-model value preference inertia. This study is the first systematic empirical challenge to the efficacy of persona prompting for value alignment. It establishes a methodological foundation and empirical evidence for evaluating LLM fairness and enabling controllable, value-guided behavior.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Philosophy and Ethics of AI: Morality & Value-based AI
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large Language Models (LLMs) exhibit non-deterministic behavior, and prompting has emerged as a primary method for steering their outputs toward desired directions. One popular strategy involves assigning a specific"persona"to the model to induce more varied and context-sensitive responses, akin to the diversity found in human perspectives. However, contrary to the expectation that persona-based prompting would yield a wide range of opinions, our experiments demonstrate that LLMs maintain consistent value orientations. In particular, we observe a persistent inertia in their responses, where certain moral and value dimensions, especially harm avoidance and fairness, remain distinctly skewed in one direction despite varied persona settings. To investigate this phenomenon systematically, use role-play at scale, which combines randomized, diverse persona prompts with a macroscopic trend analysis of model outputs. Our findings highlight the strong internal biases and value preferences in LLMs, underscoring the need for careful scrutiny and potential adjustment of these models to ensure balanced and equitable applications.

Problem

• LLMs show consistent moral values despite varied persona prompts
• Inertia in LLM responses skews harm avoidance and fairness
• Internal biases in LLMs need scrutiny for equitable applications

Innovation

• Uses role-play at scale for analysis
• Combines randomized diverse persona prompts
• Applies macroscopic trend analysis technique