ALVI Interface: Towards Full Hand Motion Decoding for Amputees Using sEMG

2025-02-28

AI Summary

This work addresses the limited motion control accuracy of prosthetic hands for upper-limb amputees. We propose a novel real-time method for decoding 20 degrees of freedom (DOF) of finger joint angles from surface electromyography (sEMG) signals at 25 Hz. Methodologically, we introduce the first integration of a VR-based virtual training environment with a lightweight Transformer architecture to establish an end-to-end sEMG-to-joint-angle mapping framework, augmented by an adaptive, low-latency closed-loop interactive interface (ALVI) enabling real-time calibration. The system operates using only eight sEMG channels and integrates VR-driven data acquisition, low-latency signal processing, and closed-loop feedback control. Offline evaluation yields a mean correlation coefficient of 0.80; in VR, it achieves sub-millimeter precision in virtual finger motion control. Key contributions are: (1) the first VR-Transformer fusion architecture designed specifically for real-time, full-hand 20-DOF angle decoding; and (2) a lightweight, online adaptive closed-loop control paradigm with built-in calibration capability.

Technology Category

• Intelligent Robots: Manipulation
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior
• Computer Vision: Low Level & Physics-based Vision

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

We present a system for decoding hand movements using surface EMG signals. The interface provides real-time (25 Hz) reconstruction of finger joint angles across 20 degrees of freedom, designed for upper limb amputees. Our offline analysis shows 0.8 correlation between predicted and actual hand movements. The system functions as an integrated pipeline with three key components: (1) a VR-based data collection platform, (2) a transformer-based model for EMG-to-motion transformation, and (3) a real-time calibration and feedback module called ALVI Interface. Using eight sEMG sensors and a VR training environment, users can control their virtual hand down to finger joint movement precision, as demonstrated in our video: youtube link.

Problem

• Decoding full hand motion for amputees using sEMG signals.
• Real-time reconstruction of finger joint angles with 20 degrees of freedom.
• Integration of VR, transformer models, and real-time calibration for precise control.

Innovation

• Real-time hand motion decoding using sEMG signals
• Transformer-based model for EMG-to-motion transformation
• VR-based data collection and real-time feedback system