Tiny-BioMoE: a Lightweight Embedding Model for Biosignal Analysis

2025-07-29

AI Summary

To address the challenges of discontinuity, subjectivity, and clinical deployability in pain assessment, this paper proposes a lightweight multimodal physiological signal analysis framework. Methodologically, we design Tiny-BioMoE—a pretrained embedding model with only 7.3 million parameters—that integrates electrodermal activity (EDA), blood volume pulse (BVP), respiration, and peripheral capillary oxygen saturation (SpO₂) signals. Leveraging bio-signal image representation and a Mixture-of-Experts (MoE) architecture, Tiny-BioMoE undergoes self-supervised pretraining on 4.4 million samples. The model enables efficient, robust cross-modal embedding extraction, achieving significant improvements in accuracy and generalization across multi-center pain recognition tasks, while maintaining low inference latency and minimal computational resource requirements. Source code and pretrained weights are publicly released, establishing a novel paradigm for real-time, wearable-enabled pain monitoring and clinical decision support.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Mixture of Experts (MoE)
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Web Mining and Content Analysis: Large pretrained models with web data
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Pain is a complex and pervasive condition that affects a significant portion of the population. Accurate and consistent assessment is essential for individuals suffering from pain, as well as for developing effective management strategies in a healthcare system. Automatic pain assessment systems enable continuous monitoring, support clinical decision-making, and help minimize patient distress while mitigating the risk of functional deterioration. Leveraging physiological signals offers objective and precise insights into a person's state, and their integration in a multimodal framework can further enhance system performance. This study has been submitted to the extit{Second Multimodal Sensing Grand Challenge for Next-Gen Pain Assessment (AI4PAIN)}. The proposed approach introduces extit{Tiny-BioMoE}, a lightweight pretrained embedding model for biosignal analysis. Trained on $4.4$ million biosignal image representations and consisting of only $7.3$ million parameters, it serves as an effective tool for extracting high-quality embeddings for downstream tasks. Extensive experiments involving electrodermal activity, blood volume pulse, respiratory signals, peripheral oxygen saturation, and their combinations highlight the model's effectiveness across diverse modalities in automatic pain recognition tasks. extit{ extcolor{blue}{The model's architecture (code) and weights are available at https://github.com/GkikasStefanos/Tiny-BioMoE.

Problem

• Develops lightweight model for biosignal-based pain assessment
• Enables automatic pain recognition using physiological signals
• Improves multimodal pain analysis with efficient embeddings

Innovation

• Lightweight pretrained embedding model for biosignals
• Trained on 4.4 million biosignal image representations
• Effective across diverse modalities in pain recognition