Not All Students Engage Alike: Multi-Institution Patterns in GenAI Tutor Use

2026-01-31

AI Summary

This study addresses the variability in student engagement with generative artificial intelligence (GenAI) tutoring tools and its potential impact on learning outcomes, an area lacking clarity across institutions and disciplines. Analyzing de-identified interaction logs from 11,406 students across 200 courses at ten universities, the authors develop the first multi-institutional analytical framework for GenAI engagement. By integrating conversation-level clustering with longitudinal behavioral analysis, they identify distinct shallow (e.g., copy-pasting, 10.4% of users) and deep engagement patterns, along with their dynamic transitions over time. Findings reveal that deep engagers flexibly adapt their strategies, whereas shallow engagers tend toward rigid usage. Notably, students at highly selective institutions exhibit a significantly stronger propensity for deep engagement, highlighting cross-institutional heterogeneity in GenAI interaction patterns.

Technology Category

• Natural Language Processing: Generation
• Humans and AI: Game Design — Procedural Content Generation & Storytelling
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Social Networks and Social Media: Generative AI / large language models and their impact on social systems
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• User Modeling, Personalization and Recommendation: Studies of user behavior, including longitudinal effects of personalized systems

Abstract

The emergence of generative artificial intelligence (GenAI) has created unprecedented opportunities to provide individualized learning support in classrooms as automated tutoring systems at scale. However, concerns have been raised that students may engage with these tools in ways that do not support learning. Moreover, student engagement with GenAI Tutors may vary across instructional contexts, potentially leading to unequal learning experiences. In this study, we utilize de-identified student interaction logs from an existing GenAI Tutor and the learning management system in which it is embedded. We systematically examined student engagement (N = 11,406) with the tool across 200 classes in ten post-secondary institutions through a two-stage pipeline: First, we identified four distinct engagement types at the conversation session level. In particular, 10.4% of them were"shallow engagement"where copy-pasting behavior was prevalent. Then, at the student level, we show that students transitioned across engagement types over time. However, students who exhibited shallow engagement with the tool were more likely to remain in this mode, whereas those who engaged deeply with the tool transitioned more flexibly across engagement types. Finally, at both the session and student levels, we show substantial heterogeneity in student engagement across institution selectivity and course disciplines. In particular, students from highly selective institutions were more likely to exhibit deep engagement. Together, our study advances the understanding of how GenAI Tutors are used in authentic educational settings and provides a framework for analyzing student engagement with GenAI Tutors, with implications for responsible implementation at scale.

Problem

• GenAI Tutor
• student engagement
• educational equity
• learning behavior
• institutional variation

Innovation

• GenAI Tutor
• student engagement patterns
• multi-institution analysis
• interaction logs
• engagement typology