More AI Assistance Reduces Cognitive Engagement: Examining the AI Assistance Dilemma in AI-Supported Note-Taking

2025-09-03

AI Summary

This study investigates the “cognitive engagement dilemma” in AI-assisted note-taking: a nonlinear relationship between AI support intensity and users’ conceptual understanding. Using a within-subjects experimental design, participants watched lecture videos and took notes under three AI assistance conditions: fully automated note generation (high automation), real-time summarization (moderate support), and transcript-only provision (low support). Results revealed significantly superior post-test performance under moderate support, whereas high automation impaired knowledge retention. Strikingly, participants subjectively preferred the fully automated condition—demonstrating a systematic misalignment between perceived utility and actual learning outcomes. This work provides the first empirical formulation and validation of the “AI-assisted dilemma” construct, identifying a critical cognitive boundary for educational AI tool design. Findings underscore that optimal learning support lies not in maximal automation but in calibrated scaffolding that preserves active cognitive engagement.

Technology Category

• Humans and AI: Intelligent User Interfaces
• Cognitive Modeling & Cognitive Systems: Analogy
• Intelligent Robots: Embodied AI

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

As AI tools become increasingly embedded in cognitively demanding tasks such as note-taking, questions remain about whether they enhance or undermine cognitive engagement. This paper examines the "AI Assistance Dilemma" in note-taking, investigating how varying levels of AI support affect user engagement and comprehension. In a within-subject experiment, we asked participants (N=30) to take notes during lecture videos under three conditions: Automated AI (high assistance with structured notes), Intermediate AI (moderate assistance with real-time summary, and Minimal AI (low assistance with transcript). Results reveal that Intermediate AI yields the highest post-test scores and Automated AI the lowest. Participants, however, preferred the automated setup due to its perceived ease of use and lower cognitive effort, suggesting a discrepancy between preferred convenience and cognitive benefits. Our study provides insights into designing AI assistance that preserves cognitive engagement, offering implications for designing moderate AI support in cognitive tasks.

Problem

• Examining AI assistance levels' impact on cognitive engagement
• Investigating how AI support affects user comprehension in note-taking
• Exploring the discrepancy between preferred convenience and cognitive benefits

Innovation

• Intermediate AI assistance optimizes cognitive engagement
• Real-time summary support enhances comprehension outcomes
• Moderate AI balancing user preference and performance