Assist-As-Needed: Adaptive Multimodal Robotic Assistance for Medication Management in Dementia Care

2025-10-08

AI Summary

Dementia patients exhibit progressive decline in medication management capacity, yet existing assistive technologies lack dynamic adaptability, compromising patient autonomy and exacerbating caregiver burden. This study proposes an adaptive, multimodal medication assistance framework implemented on the Pepper robot. Grounded in occupational therapy principles, it employs a hierarchical, progression-based intervention strategy; integrates large language models with multimodal perception to enable dynamic mode switching—from voice reminders, to voice-and-gesture cues, to physical navigation and step-by-step guidance. Preliminary validation was conducted in controlled experiments with healthy adults and dementia care stakeholders. Results demonstrate strong usability, comprehensibility, and adaptive feedback responsiveness. To our knowledge, this is the first systematic application of dignity-centered, personalized human-robot collaboration to dementia-related medication support. The framework establishes a novel paradigm for clinical adaptability in intelligent assistive technology design.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Humans and AI: Intelligent User Interfaces
• Cognitive Modeling & Cognitive Systems: Adaptive Behavior

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search

Abstract

People living with dementia (PLWDs) face progressively declining abilities in medication management-from simple forgetfulness to complete task breakdown-yet most assistive technologies fail to adapt to these changing needs. This one-size-fits-all approach undermines autonomy, accelerates dependence, and increases caregiver burden. Occupational therapy principles emphasize matching assistance levels to individual capabilities: minimal reminders for those who merely forget, spatial guidance for those who misplace items, and comprehensive multimodal support for those requiring step-by-step instruction. However, existing robotic systems lack this adaptive, graduated response framework essential for maintaining PLWD independence. We present an adaptive multimodal robotic framework using the Pepper robot that dynamically adjusts assistance based on real-time assessment of user needs. Our system implements a hierarchical intervention model progressing from (1) simple verbal reminders, to (2) verbal + gestural cues, to (3) full multimodal guidance combining physical navigation to medication locations with step-by-step verbal and gestural instructions. Powered by LLM-driven interaction strategies and multimodal sensing, the system continuously evaluates task states to provide just-enough assistance-preserving autonomy while ensuring medication adherence. We conducted a preliminary study with healthy adults and dementia care stakeholders in a controlled lab setting, evaluating the system's usability, comprehensibility, and appropriateness of adaptive feedback mechanisms. This work contributes: (1) a theoretically grounded adaptive assistance framework translating occupational therapy principles into HRI design, (2) a multimodal robotic implementation that preserves PLWD dignity through graduated support, and (3) empirical insights into stakeholder perceptions of adaptive robotic care.

Problem

• Developing adaptive robotic assistance for dementia medication management
• Creating multimodal support that adjusts to individual cognitive decline
• Implementing hierarchical intervention to preserve autonomy in dementia care

Innovation

• Adaptive multimodal robotic framework using Pepper robot
• Hierarchical intervention model with three assistance levels
• LLM-driven interaction strategies with multimodal sensing