Critical Thinking with Generative AI: A Constraint-First Design Pilot of a Thinking-Partner Intervention

2026-09-29

AI Summary

This study addresses the lack of critical evaluation regarding how generative AI integration affects critical thinking in classrooms. Through a design-based research pilot, ChatGPT was introduced as a thinking partner in an undergraduate course. Methodologically, we propose a “constraint-first” framework for AI-assisted critical thinking, employing mixed methods that combine quantitative assessments, journal analysis, and qualitative coding to reveal students’ positioning patterns during interactions with large language models. Results indicate that the intervention significantly improved statistical learning and AI literacy; however, standardized critical thinking scores remained unchanged, corroborating the phenomenon of insufficient human–AI dialogue. This work provides a novel framework and empirical evidence for the effective integration of AI in educational settings.

Abstract

Generative AI (GenAI) tools entered higher education classrooms faster than the field was able to study their effects on learning. One concern is that GenAI may displace the critical thinking and AI literacy that students will need after graduation. This paper reports a Design-Based Research pilot of a GenAI-assisted critical thinking framework, in which ChatGPT was used as a thinking partner in an undergraduate research methods and statistics course during Spring 2025 (N = 14). The mixed-methods design combined pre- and post-intervention measures of statistical learning (AASCDM), AI literacy (MAILS), and critical thinking (WGCTA) with instructor field notes, student artifacts, and student-AI interaction logs. Pre-post tests showed gains on every AASCDM dimension and on eight of nine MAILS dimensions, while WGCTA percentiles did not change. Qualitative analysis identified four themes: the ways students positioned the LLM (as answer generator, validator, or co-thinker); the depth of student engagement (procedural vs. conceptual); occasional humanizing of the tool; and the role of curriculum design in shaping each of the prior three. Read together, the findings indicate that one semester of GenAI-assisted instruction can move domain learning and self-reported AI literacy but does not move standardized critical thinking, and that the modal student-LLM relationship is one of validation instead of dialogue. We end with design principles for the next iteration of the framework and implications for research on adaptive and personalized learning.

Problem

• Generative AI
• Critical Thinking
• AI Literacy
• Higher Education
• Thinking Partner

Innovation

• Generative AI
• Critical Thinking
• Thinking Partner
• Constraint-First Design
• Design-Based Research