🤖 AI Summary
Existing time-series classification models (e.g., PatchTST) emphasize low-frequency temporal dynamics while neglecting critical high-frequency components, resulting in poor discrimination of subtle transient patterns. To address this, we propose a dual-stream hybrid architecture: building upon the PatchTST backbone, we introduce a learnable wavelet branch that explicitly captures complementary high-frequency information via deep wavelet packet decomposition (WPD); additionally, we design a learnable generalized mean (GeM) pooling layer to enhance discriminability of time-frequency features. This work is the first to seamlessly integrate learnable wavelet representations with Transformer-based temporal modeling. Our method achieves 93.38% mean accuracy on UCI-HAR, outperforming state-of-the-art models including PatchTST. Ablation studies confirm the necessity and effectiveness of both the wavelet branch and the learnable GeM pooling.
📝 Abstract
Transformers have become state-of-the-art (SOTA) for time-series classification, with models like PatchTST demonstrating exceptional performance. These models rely on patching the time series and learning relationships between raw temporal data blocks. We argue that this approach is blind to critical, non-obvious high-frequency information that is complementary to the temporal dynamics. In this letter, we propose Hi-WaveTST, a novel Hybrid architecture that augments the original temporal patch with a learnable, High-Frequency wavelet feature stream. Our wavelet stream uses a deep Wavelet Packet Decomposition (WPD) on each patch and extracts features using a learnable Generalized Mean (GeM) pooling layer. On the UCI-HAR benchmark dataset, our hybrid model achieves a mean accuracy of 93.38 percent plus-minus 0.0043, significantly outperforming the SOTA PatchTST baseline (92.59 percent plus-minus 0.0039). A comprehensive ablation study proves that every component of our design-the hybrid architecture, the deep high-frequency wavelet decomposition, and the learnable GeM pooling-is essential for this state-of-the-art performance.