Assistant or Actor? Student Trust, Control, and Delegation Regret When Using a General-Purpose AI Agent

2026-05-14

arXiv.org

AI Summary

This study addresses the trust crisis and "delegation regret" that arise when general-purpose AI agents transition from question-answering to action execution, as users cannot anticipate the action space. Through a controlled experiment in which students used OpenClaw to perform daily tasks, combined with Likert-scale surveys and thematic coding, the authors quantitatively analyze trust calibration and behavioral preferences under varying risk and reversibility conditions. This work introduces the concept of delegation regret, revealing that irreversibility and external visibility drive trust withdrawal more strongly than high risk alone, while users calibrate trust at task-level granularity. Furthermore, it demonstrates that execution without previews induces persistent regret. Accordingly, the authors recommend agent architectures that expose action boundaries, support autonomous strategies, and decouple consultation from execution.

Abstract

When AI agents shift from answering questions to taking actions, users face a new problem: deciding what to delegate, to a system whose action space they cannot fully anticipate. We call the resulting dissatisfaction delegation regret, a pattern in which users regret not that the agent erred, but that it acted beyond what they would have authorized. In a controlled study, 20 university students completed five common daily tasks using OpenClaw, a general-purpose AI agent, across tasks chosen to vary in privacy, stakes, and reversibility. For each task we measured trust, perceived control, transparency, supervision burden, and approval preference on 5-point Likert scales, and collected free-text reflections analyzed through thematic coding. Three findings emerged. First, participants calibrated trust per task rather than per agent: they granted wide autonomy for advisory and low-stakes tasks but demanded confirmation for irreversible, externally visible actions. Second, irreversibility combined with external visibility, rather than stakes alone, appeared to drive trust withdrawal: the moderate-stakes email task triggered the sharpest drop in trust (M = 3.10) and the highest demand for approval (M = 4.65), whereas a high-stakes but verifiable task did not produce the same response. Third, delegation regret appeared consistently when the agent executed actions without preview, even when the output was rated as successful. We discuss implications for agent designs that expose action boundaries, support per-task autonomy policies, and separate advisory output from agentic execution.

Problem

• AI agent
• delegation regret
• user trust
• perceived control
• task delegation

Innovation

• Delegation Regret
• Trust Calibration
• AI Agent
• Perceived Control
• Action Boundaries