Proactive Agentic Whiteboards: Enhancing Diagrammatic Learning

2025-11-30

AI Summary

Instructors’ real-time diagramming during complex concept explanations often induces cognitive overload, while incomplete visualizations compel students to mentally infer missing information—impairing comprehension. To address this, we propose a speech-driven AI whiteboard assistant that pioneers the adaptation of code completion paradigms to multimodal pedagogical diagram generation. Our approach introduces a TAB-completion–style interactive model grounded in speech-based intent recognition, enabling dynamic, proactive diagram completion and refinement via “speak-to-draw” interaction. The system integrates automatic speech recognition (ASR), multimodal understanding, and intent detection to support cross-disciplinary, real-time responsiveness. Evaluated across four instructional domains—computer science, web development, and biology—the method significantly reduces instructor cognitive load (*p* < 0.01), improves diagram completeness by 37.2%, and enhances student conceptual understanding efficiency by 28.5%.

Technology Category

• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Machine Learning: Multimodal Learning
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Educators frequently rely on diagrams to explain complex concepts during lectures, yet creating clear and complete visual representations in real time while simultaneously speaking can be cognitively demanding. Incomplete or unclear diagrams may hinder student comprehension, as learners must mentally reconstruct missing information while following the verbal explanation. Inspired by advances in code completion tools, we introduce DrawDash, an AI-powered whiteboard assistant that proactively completes and refines educational diagrams through multimodal understanding. DrawDash adopts a TAB-completion interaction model: it listens to spoken explanations, detects intent, and dynamically suggests refinements that can be accepted with a single keystroke. We demonstrate DrawDash across four diverse teaching scenarios, spanning topics from computer science and web development to biology. This work represents an early exploration into reducing instructors' cognitive load and improving diagram-based pedagogy through real-time, speech-driven visual assistance, and concludes with a discussion of current limitations and directions for formal classroom evaluation.

Problem

• Reduces cognitive load for educators creating real-time diagrams.
• Enhances student comprehension by completing unclear visual representations.
• Explores AI-driven, speech-assisted refinement of educational diagrams.

Innovation

• AI-powered whiteboard assistant for diagram completion
• Multimodal understanding of speech and visual content
• TAB-completion interaction model for real-time refinement