Promoting Critical Thinking With Domain-Specific Generative AI Provocations

2026-03-20

AI Summary

This work addresses how generative AI can effectively foster users’ critical thinking in domain-specific contexts while mitigating its inherent risks. The authors propose the concept of “productive friction” and design a domain-adaptive, dynamic provocation mechanism that leverages user-response-driven prompt generation and human-AI collaborative interaction. Implemented in two knowledge-work scenarios—art interpretation (ArtBot) and AI privacy (Privy)—the system guides users to clarify their positions and develop reasoned arguments. It dynamically tailors its provocation strategies based on users’ contributions, preferences, and expertise levels, thereby overcoming the limitations of static prompting. User studies demonstrate that this adaptive provocation approach, grounded in domain knowledge and contingent on active user engagement, significantly enhances critical thinking, confirming both the feasibility and necessity of this interactive paradigm.

Technology Category

Humans and AI: Intelligent User InterfacesNatural Language Processing: Prompt Engineering / PromptingCognitive Modeling & Cognitive Systems: Computational Creativity

Application Category

Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applicationsSocial Networks and Social Media: Generative AI / large language models and their impact on social systemsSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

The evidence on the effects of generative AI (GenAI) on critical thinking is mixed, with studies suggesting both potential harms and benefits depending on its implementation. Some argue that AI-driven provocations, such as questions asking for human clarification and justification, are beneficial for eliciting critical thinking. Drawing on our experience designing and evaluating two GenAI-powered tools for knowledge work, ArtBot in the domain of fine art interpretation and Privy in the domain of AI privacy, we reflect on how design decisions shape the form and effectiveness of such provocations. Our observations and user feedback suggest that domain-specific provocations, implemented through productive friction and interactions that depend on user contribution, can meaningfully support critical thinking. We present participant experiences with both prototypes and discuss how supporting critical thinking may require moving beyond static provocations toward approaches that adapt to user preferences and levels of expertise.

Problem

• generative AI
• critical thinking
• domain-specific provocations
• productive friction
• user contribution

Innovation

• generative AI
• critical thinking
• domain-specific provocations
• productive friction
• adaptive interaction