Functional Near-Infrared Spectroscopy (fNIRS) Analysis of Interaction Techniques in Touchscreen-Based Educational Gaming

2024-05-14

AI Summary

This study investigates the impact of two input modalities—finger touch versus stylus use—in touchscreen educational games on learners’ cognitive load. Using functional near-infrared spectroscopy (fNIRS), we measured real-time changes in prefrontal oxyhemoglobin concentration (ΔHbO), complemented by subjective self-reports and behavioral metrics. We propose and quantify two novel neurocognitive indices: Relative Neural Involvement (RNI) and Relative Neural Efficiency (RNE). Results show that finger touch significantly reduces ΔHbO and RNI (p < 0.01) while increasing RNE compared to stylus use—indicating lower cognitive effort and higher neural efficiency. Subjective assessments corroborate these findings, reporting greater intuitiveness, reduced frustration, and superior user experience with finger touch. This work provides empirically grounded, neurophysiological evidence for educational human–computer interaction and establishes a new analytical framework for evaluating how input modalities shape cognitive resource allocation.

Technology Category

• Humans and AI: Interaction Techniques and Devices
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Application Domains: Humanities & Computational Social Science

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Educational games enhance learning experiences by integrating touchscreens, making interactions more engaging and intuitive for learners. However, the cognitive impacts of educational gameplay input modalities--such as the hand and stylus technique--are unclear. We compared the experience of using hands vs. stylus for touchscreens while playing an educational game by analyzing oxygenated hemoglobin ($Delta HbO$) collected by functional Near-Infrared Spectroscopy (fNIRS) and self-reported measures. In addition, we measured the hand versus the stylus modalities of the task and calculated the relative neural engagement (RNE) and relative neural involvement (RNI) using the ($Delta HbO$) and the quiz score. Our findings show that the hand condition had a significantly lower ($Delta HbO$) and RNI, yet higher RNE than the stylus condition, indicating the requirement of less cognitive effort. Additionally, self-reported measures reported statistically significant differences. The results suggest that hand-based input is more intuitive, less cognitively demanding, and less frustrating. Conversely, the use of a stylus required higher cognitive effort due to the cognitive balance of controlling the pen and answering questions.

Problem

• cognitive impacts of input modalities
• hand vs. stylus in educational games
• measuring neural engagement with fNIRS

Innovation

• fNIRS measures cognitive impact
• Compare hand vs stylus input
• Calculate neural engagement indices