Exploring Learners'Expectations and Engagement When Collaborating with Constructively Controversial Peer Agents

2026-01-20

AI Summary

This study investigates how large language model (LLM)-driven peer agents can effectively introduce constructive controversy to enhance learning in asynchronous online courses. Employing a mixed-methods design (N=144), it compares agents constrained by norms of constructive controversy with those operating under looser, goal-oriented guidelines, while also examining the impact of design transparency—specifically, whether or not the agent’s artificial nature is disclosed—on learner experience. As the first application of constructive controversy theory to LLM-based peer agents, the research reveals that learners’ values significantly moderate their preferences for controversy styles: those prioritizing control favor non-normative agents they can disengage from at will, whereas those seeking intellectual challenge prefer norm-guided agents. Although transparency improves comprehension of the agent’s role, it concurrently diminishes perceived trust in its capabilities.

Technology Category

• Humans and AI: Learning Human Values and Preferences
• Multiagent Systems: Mechanism Design
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Peer agents can supplement real-time collaborative learning in asynchronous online courses. Constructive Controversy (CC) theory suggests that humans deepen their understanding of a topic by confronting and resolving controversies. This study explores whether CC's benefits apply to LLM-based peer agents, focusing on the impact of agents'disputatious behaviors and disclosure of agents'behavior designs on the learning process. In our mixed-method study (n=144), we compare LLMs that follow detailed CC guidelines (regulated) to those guided by broader goals (unregulated) and examine the effects of disclosing the agents'design to users (transparent vs. opaque). Findings show that learners'values influence their agent interaction: those valuing control appreciate unregulated agents'willingness to cease push-back upon request, while those valuing intellectual challenges favor regulated agents for stimulating creativity. Additionally, design transparency lowers learners'perception of agents'abilities. Our findings lay the foundation for designing effective collaborative peer agents in isolated educational settings.

Problem

• Constructive Controversy
• peer agents
• learner engagement
• LLM-based agents
• design transparency

Innovation

• Constructive Controversy
• LLM-based peer agents
• design transparency
• learner engagement
• asynchronous online learning