Design Probes for AI-Driven AAC: Addressing Complex Communication Needs in Aphasia

2025-04-13

AI Summary

Aphasia exhibits high heterogeneity in communication impairments, posing significant challenges for designing effective augmentative and alternative communication (AAC) systems. Method: This study proposes an AI-capability-centered “research-through-design” two-phase paradigm, developing four multimodal AI-augmented AAC prototypes. Technical approaches integrate text/image/speech generation, real-time automatic speech recognition (ASR) and text-to-speech (TTS), personalized language model fine-tuning, and user-intent co-calibration. We introduce four novel interaction mechanisms: visual verification, syntactic construction support, error correction, and cognitive load reduction. Contribution/Results: In a user study with 11 persons with aphasia (PWAs), the prototypes significantly improved communicative intelligibility and perceived autonomy. Key challenges—including AI-user intent misalignment—were identified, yielding 12 evidence-based design principles for clinically deployable AAC. The core innovation lies in deeply embedding AI-native capabilities into a human–AI collaborative communication loop, transforming AAC from a passive output tool into a cognitively supportive conversational partner capable of pre-utterance preparation and real-time adaptation.

Technology Category

• Humans and AI: AI for Accessibility
• Cognitive Modeling & Cognitive Systems: Affective Computing
• Multiagent Systems: Agent Communication

Application Category

• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

AI offers key advantages such as instant generation, multi-modal support, and personalized adaptability - potential that can address the highly heterogeneous communication barriers faced by people with aphasia (PWAs). We designed AI-enhanced communication tools and used them as design probes to explore how AI's real-time processing and generation capabilities - across text, image, and audio - can align with PWAs' needs in real-time communication and preparation for future conversations respectively. Through a two-phase"Research through Design"approach, eleven PWAs contributed design insights and evaluated four AI-enhanced prototypes. These prototypes aimed to improve communication grounding and conversational agency through visual verification, grammar construction support, error correction, and reduced language processing load. Despite some challenges, such as occasional mismatches with user intent, findings demonstrate how AI's specific capabilities can be advantageous in addressing PWAs' complex needs. Our work contributes design insights for future Augmentative and Alternative Communication (AAC) systems.

Problem

• AI-driven AAC tools for aphasia communication barriers
• Real-time multi-modal AI support for PWAs' needs
• Enhancing conversational agency with AI prototypes

Innovation

• AI-enhanced tools for real-time multi-modal communication
• Design probes with visual and grammar support
• Reduced processing load via error correction