EMG-GPT: Predictive Pretraining on Residual-Quantized EMG Tokens for Hand Pose Estimation

2026-10-04

AI Summary

This study addresses the lack of transferable self-supervised representations for surface electromyography (sEMG)-based continuous hand pose estimation by proposing the EMG-GPT framework. This method introduces, for the first time, a combination of frozen residual vector quantization (RVQ) and a causal Transformer to enable self-supervised pretraining on raw EMG signals. By preserving codebook geometric structures while learning sEMG temporal dynamics through deep autoregressive prediction of future discrete tokens, the framework effectively captures signal characteristics without supervision. Experimental results demonstrate that the proposed model achieves highly competitive performance in both hand pose regression and tracking tasks. These findings validate the feasibility of acquiring cross-task transferable representations relying solely on pure EMG pretraining, offering a promising direction for sEMG-based interaction systems.

Abstract

Surface electromyography (sEMG) is a low-power, cost-effective biosignal for hand-pose estimation and gesture classification. In this work, we examine whether self-supervised pretraining on sEMG can yield transferable representations for continuous hand-pose estimation. We introduce EMG-GPT, a causal transformer-based model that operates on discrete sEMG representations from a frozen residual vector quantization (RVQ) tokenizer and learns temporal dynamics through depth-autoregressive future-code prediction. The model combines within-frame integration with causal temporal modeling while preserving the geometry of the pretrained codebook. EMG-GPT shows competitive results in both Regression and Tracking tasks, supporting EMG-only pretraining as a viable approach for learning transferable sEMG representations.

Problem

• surface electromyography
• hand-pose estimation
• self-supervised pretraining
• transferable representations

Innovation

• Self-supervised pretraining
• Residual vector quantization
• Causal transformer
• Surface electromyography
• Hand pose estimation