Eggly

2023-06-12

Proceedings of the ACM on Interactive Mobile Wearable and Ubiquitous Technologies

AI Summary

Clinical neurofeedback training (NFT) for children with autism spectrum disorder (ASD) faces challenges in accessibility, mobility, and engagement due to its reliance on stationary setups and lack of gamified, context-aware design. Method: We propose the first mobile augmented reality (AR) neurofeedback gaming framework tailored for ASD children, leveraging consumer-grade EEG headsets and tablets. The system integrates real-time theta/beta ratio decoding, contextual AR visual feedback, and adaptive difficulty adjustment within a lightweight Unity-based AR rendering pipeline (ARKit/ARCore) and closed-loop neurofeedback control. Contribution/Results: Two field studies—a single-session trial and a three-week multi-session intervention—demonstrated significantly improved attention and engagement; all participants completed the training. Qualitative feedback indicated high enjoyment and immersion, while quantitative analysis revealed a consistent trend toward normalized theta/beta ratios, validating the feasibility and efficacy of mobile AR-NFT for ASD intervention.

Technology Category

• Cognitive Modeling & Cognitive Systems: Adaptive Behavior
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Multiagent Systems: Adversarial Agents

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Search and Retrieval-Augmented AI: Personalized, context-aware and across-device search
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Autism Spectrum Disorder (ASD) is a neurodevelopmental disorder that affects how children communicate and relate to other people and the world around them. Emerging studies have shown that neurofeedback training (NFT) games are an effective and playful intervention to enhance social and attentional capabilities for autistic children. However, NFT is primarily available in a clinical setting that is hard to scale. Also, the intervention demands deliberately-designed gamified feedback with fun and enjoyment, where little knowledge has been acquired in the HCI community. Through a ten-month iterative design process with four domain experts, we developed Eggly, a mobile NFT game based on a consumer-grade EEG headband and a tablet. Eggly uses novel augmented reality (AR) techniques to offer engagement and personalization, enhancing their training experience. We conducted two field studies (a single-session study and a three-week multi-session study) with a total of five autistic children to assess Eggly in practice at a special education center. Both quantitative and qualitative results indicate the effectiveness of the approach as well as contribute to the design knowledge of creating mobile AR NFT games.

Problem

• Develop mobile neurofeedback training games for autistic children.
• Enhance social and attentional skills through gamified feedback.
• Use augmented reality for engagement and personalization in training.

Innovation

• Mobile AR neurofeedback training for autism
• Consumer-grade EEG headband and tablet integration
• Iterative design with domain experts