ChatGPT's advice drives moral judgments with or without justification

2025-01-03

AI Summary

Users increasingly rely on AI chatbots (e.g., ChatGPT) for moral decision-making, yet the psychological mechanisms underlying such reliance remain poorly understood. Method: Through controlled behavioral experiments comparing AI versus human moral advisors—with systematic variation in justification provision and source attribution—we examine adoption patterns of ethical recommendations. Contribution/Results: We provide the first empirical evidence that users adopt AI-generated moral advice not based on argument quality, but as a low-cost strategy to evade moral responsibility—termed “moral offloading.” Adoption rates show no significant difference across conditions (justified vs. unjustified advice; AI vs. human source), while AI’s accessibility amplifies this tendency. Crucially, moral offloading—not perceived technical credibility—emerges as the primary driver of advice adoption. These findings underscore the urgent need to integrate ethical literacy with digital literacy in education and policy, offering critical empirical grounding for AI ethics governance frameworks.

Technology Category

• Philosophy and Ethics of AI: Morality & Value-based AI
• Humans and AI: Learning Human Values and Preferences
• Natural Language Processing: Ethics — Bias, Fairness, Transparency & Privacy

Application Category

• User Modeling, Personalization and Recommendation: Accountability, Transparency, and Ethics for personalization
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Why do users follow moral advice from chatbots? A chatbot is not an authoritative moral advisor, but it can generate seemingly plausible arguments. Users do not follow reasoned more readily than unreasoned advice, though, we find in an experiment. However, this is also true if we attribute advice to a moral advisor, not a chatbot. Hence, it seems that advice offers users a cheap way to escape from a moral dilemma. This is a concern that chatbots do not raise, but they exacerbate it as they make advice easily accessible. We conclude that it takes ethical in addition to digital literacy to harness users against moral advice from chatbots.

Problem

• Ethical Advice
• Moral Judgment
• Robot Assistance

Innovation

• Moral Advice Perception
• Robot Influence Parity
• Ethical Awareness Necessity