Technology Category
Machine Learning: Time-Series/Data StreamsComputer Vision: Medical and Biological ImagingCognitive Modeling & Cognitive Systems: Neural Spike Coding
WaveFormer: Wavelet Embedding Transformer for Biomedical Signals
🤖 AI Summary
Standard Transformers struggle to effectively model long-range dependencies, complex temporal dynamics, and multi-scale frequency characteristics inherent in biomedical signals. To address this limitation, this work proposes WaveFormer, a novel Transformer architecture that integrates wavelet decomposition to jointly model time-frequency features during token embedding and positional encoding. The key innovations include generating frequency-aware tokens via multi-channel discrete wavelet transform (DWT) and introducing dynamic wavelet positional encoding (DyWPE) to adaptively capture the temporal structure of signals. Evaluated on eight datasets spanning human activity recognition and electroencephalogram (EEG) analysis, WaveFormer demonstrates competitive performance, validating its effectiveness in modeling multi-scale frequency information in biomedical time-series data.
Technology Category
Application Category
📝 Abstract
Biomedical signal classification presents unique challenges due to long sequences, complex temporal dynamics, and multi-scale frequency patterns that are poorly captured by standard transformer architectures. We propose WaveFormer, a transformer architecture that integrates wavelet decomposition at two critical stages: embedding construction, where multi-channel Discrete Wavelet Transform (DWT) extracts frequency features to create tokens containing both time-domain and frequency-domain information, and positional encoding, where Dynamic Wavelet Positional Encoding (DyWPE) adapts position embeddings to signal-specific temporal structure through mono-channel DWT analysis. We evaluate WaveFormer on eight diverse datasets spanning human activity recognition and brain signal analysis, with sequence lengths ranging from 50 to 3000 timesteps and channel counts from 1 to 144. Experimental results demonstrate that WaveFormer achieves competitive performance through comprehensive frequency-aware processing. Our approach provides a principled framework for incorporating frequency-domain knowledge into transformer-based time series classification.
Problem
Research questions and friction points this paper is trying to address.
biomedical signal classification
long sequences
temporal dynamics
multi-scale frequency patterns
transformer architectures
Innovation
Methods, ideas, or system contributions that make the work stand out.
Wavelet Embedding
Discrete Wavelet Transform (DWT)
Dynamic Wavelet Positional Encoding
Frequency-aware Transformer
Biomedical Signal Classification
🔎 Similar Papers
No similar papers found.
