DyWPE: Signal-Aware Dynamic Wavelet Positional Encoding for Time Series Transformers

2025-09-18

AI Summary

Traditional position encodings are signal-agnostic and struggle to model multiscale non-stationary dynamics in time series. To address this, we propose Dynamic Wavelet Position Encoding (DWPE), the first position encoding framework for Transformers that incorporates the Discrete Wavelet Transform (DWT) to generate signal-aware, dynamically adaptive positional embeddings leveraging input-specific multiscale time-frequency features. DWPE overcomes the representational limitations of fixed sinusoidal encodings in time-frequency domains, substantially enhancing modeling capacity for complex temporal structures. Extensive experiments across 10 benchmark datasets demonstrate that DWPE achieves an average relative performance gain of 9.1% on biomedical signal tasks while maintaining computational efficiency. Our core contributions are: (i) the first DWT-based, signal-driven position encoding framework; (ii) a multiscale, dynamic, and learnable positional representation; and (iii) a favorable trade-off between accuracy and inference efficiency.

Technology Category

Machine Learning: Time-Series/Data StreamsCognitive Modeling & Cognitive Systems: Neural Spike CodingKnowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphsResponsible Web: Data and user privacy-enhancing technologies for the WebSearch and Retrieval-Augmented AI: Efficiency and scalability of Web search engines

Abstract

Existing positional encoding methods in transformers are fundamentally signal-agnostic, deriving positional information solely from sequence indices while ignoring the underlying signal characteristics. This limitation is particularly problematic for time series analysis, where signals exhibit complex, non-stationary dynamics across multiple temporal scales. We introduce Dynamic Wavelet Positional Encoding (DyWPE), a novel signal-aware framework that generates positional embeddings directly from input time series using the Discrete Wavelet Transform (DWT). Comprehensive experiments in ten diverse time series datasets demonstrate that DyWPE consistently outperforms eight existing state-of-the-art positional encoding methods, achieving average relative improvements of 9.1% compared to baseline sinusoidal absolute position encoding in biomedical signals, while maintaining competitive computational efficiency.

Problem

• Addressing signal-agnostic positional encoding in transformers
• Capturing multi-scale non-stationary dynamics in time series
• Generating signal-aware embeddings using Discrete Wavelet Transform

Innovation

• Signal-aware positional encoding using wavelets
• Dynamic Wavelet Transform for time series embedding
• Outperforms existing methods with 9.1% improvement