adaptive wavelet feature extraction

Traditional methods for nonstationary time series modeling suffer from inaccurate dynamic characterization due to their assumption of stationary statistical properties. To address this, we propose AdaWaveNet—a novel adaptive wavelet network based on a learnable lifting scheme. Its core innovation lies in the first end-to-end integration of the lifting wavelet transform into a deep neural network, enabling data-driven, multiscale wavelet decomposition and reconstruction while overcoming the representational limitations of fixed wavelet bases. The model adopts a differentiable encoder–decoder architecture, unifying support for forecasting, missing value imputation, and super-resolution. Joint multitask training further enhances generalization. Evaluated across 10 benchmark datasets and three task categories, AdaWaveNet consistently outperforms state-of-the-art methods, achieving significant improvements in forecasting accuracy, imputation robustness, and super-resolution quality.

This work addresses the challenge of spectral bias in machine learning models for dynamical system modeling, which often leads to inaccurate representation of high-frequency components and instability in long-term predictions. To mitigate this issue, the authors propose the Multi-Scale Wavelet Transformer (MSWT), a novel operator learning framework that operates in the wavelet domain. By integrating wavelet-preserving downsampling and cross-scale wavelet attention mechanisms, MSWT explicitly separates and retains both high- and low-frequency information. This approach effectively alleviates spectral bias and significantly enhances the model’s capacity to capture high-frequency structures. Extensive experiments on chaotic dynamical systems and ERA5 climate reanalysis data demonstrate substantial reductions in prediction error and climatic bias, yielding improved long-term stability and spectral fidelity.

Existing discrete wavelet transform (DWT)-based single-image super-resolution methods struggle to model inter-scale dependencies among frequency subbands, leading to artifacts and structural inconsistencies in reconstructed images. To address this, we propose the Multi-level Wavelet Spectral Diffusion Transformer (MW-DiT), which innovatively integrates multi-level DWT, pyramid tokenization, and a dual-decoder architecture to explicitly capture cross-scale spectral correlations within joint spatial-frequency representations—thereby alleviating high-low frequency misalignment. Leveraging diffusion processes as a prior, MW-DiT employs Transformer-based long-range dependency modeling and multi-scale feature co-optimization. Extensive experiments on standard benchmarks (Set5, Set14, Urban100) demonstrate significant improvements in perceptual quality and fidelity: PSNR and SSIM scores are competitive with state-of-the-art methods, while visual results exhibit richer texture details and more natural structural consistency.

This work addresses the challenge of multi-scale time series forecasting, where modeling cross-scale patterns and maintaining computational efficiency are often at odds. The authors propose AWGformer, a novel architecture that integrates adaptive wavelet decomposition (AWDM) with a frequency-aware multi-head attention mechanism (FAMA), enabling dynamic selection of wavelet bases guided by signal characteristics and facilitating interaction among multi-band features. Coupled with cross-scale feature fusion (CSFF) and a hierarchical prediction network (HPN), AWGformer forms an end-to-end trainable framework supported by theoretical convergence guarantees. Extensive experiments on multiple benchmark datasets demonstrate that AWGformer significantly outperforms existing methods, achieving superior prediction accuracy and robustness—particularly in multi-scale and non-stationary scenarios.

To address the high computational complexity (O(n²)) and substantial memory overhead of self-attention in Transformers, this paper proposes Multi-Domain Fourier-Wavelet Attention (MDFWA)—a self-attention-free token-mixing mechanism operating in the frequency domain. Methodologically, MDFWA jointly leverages discrete Fourier transform and continuous wavelet transform, integrating frequency-domain causal masking and learnable frequency bases to achieve synergistic global spectral modeling and local temporal awareness. We present the first rigorous derivation of its sub-quadratic time and memory complexity (O(n log n)) and its differentiable gradient formulation, and introduce an adaptive scale-selection strategy. Evaluated on PubMed abstract generation, the model achieves significant performance gains while reducing memory consumption by 42% and accelerating training by 2.3×. This work establishes a novel paradigm for efficient long-sequence modeling.

Existing wavelet-based time series forecasting methods over-rely on low-frequency components while neglecting the critical role of high-frequency details in prediction accuracy. To address this, we propose WaveTuner—a novel framework enabling full-spectrum subband co-optimization in the wavelet domain. It introduces an adaptive wavelet refinement module to generate time-frequency coefficients and a dynamic routing mechanism to learn subband-specific weights. Furthermore, we design a multi-branch, specialized network based on the Kolmogorov–Arnold Network (KAN) to model frequency-band-specific features, jointly capturing global trends and local dynamics. Extensive experiments across eight real-world datasets demonstrate that WaveTuner significantly enhances multi-scale modeling capability and achieves state-of-the-art (SOTA) performance in long-term forecasting tasks.

Traditional fixed analysis transforms struggle to capture the sparse structures inherent to specific signal classes due to their lack of data adaptivity. This work proposes an explicitly conditioned doubly-sparse transform that multiplies a fixed, well-conditioned matrix with a data-adaptive sparse component, thereby introducing controllable data adaptivity while preserving fast and stable computation. We devise a structured learning approach incorporating condition number control to balance generalization and approximation accuracy, and develop a novel closed-form projection operator within an inexact proximal framework for efficient optimization. The resulting method achieves state-of-the-art performance in doubly-sparse transform learning, significantly reducing computational cost compared to dense variants, converging faster, and more effectively avoiding poor local minima.

Standard Transformers struggle to effectively model long-range dependencies, complex temporal dynamics, and multi-scale frequency characteristics inherent in biomedical signals. To address this limitation, this work proposes WaveFormer, a novel Transformer architecture that integrates wavelet decomposition to jointly model time-frequency features during token embedding and positional encoding. The key innovations include generating frequency-aware tokens via multi-channel discrete wavelet transform (DWT) and introducing dynamic wavelet positional encoding (DyWPE) to adaptively capture the temporal structure of signals. Evaluated on eight datasets spanning human activity recognition and electroencephalogram (EEG) analysis, WaveFormer demonstrates competitive performance, validating its effectiveness in modeling multi-scale frequency information in biomedical time-series data.

Existing scalable kernel methods struggle to effectively model nonstationary processes—those exhibiting complex patterns whose statistical properties vary with input location. This work proposes the Random Wavelet Features (RWF) framework, which extends random feature methods to nonstationary settings for the first time. By sampling from wavelet families to construct explicit feature maps, RWF leverages the localization and multiresolution properties of wavelets to enable efficient and scalable approximation of nonstationary kernels. Theoretical analysis guarantees that the resulting kernels are positive definite, unbiased, and uniformly convergent. Empirical evaluations demonstrate that RWF outperforms conventional stationary random feature approaches across multiple synthetic and real-world datasets, achieving a superior trade-off between accuracy and computational efficiency compared to more complex models such as deep Gaussian processes.

This work addresses the rate-distortion performance bottleneck in learned image compression by proposing a channel-wise wavelet-domain Transformer architecture. The method integrates channel-wise wavelet transforms into windowed spatial self-attention, performing query, key, and value projections directly in the wavelet domain. It further incorporates channel-wise wavelet packet decomposition to optimize slice-based autoregressive entropy modeling, effectively sparsifying inter-channel covariance structures. Experimental results demonstrate significant improvements over state-of-the-art approaches, achieving BD-rate gains of −17.82%, −19.15%, and −22.56% on the Kodak, CLIC Professional Validation, and Tecnick test sets, respectively.