whitening transformation

Traditional fixed analysis transforms struggle to capture the sparse structures inherent to specific signal classes due to their lack of data adaptivity. This work proposes an explicitly conditioned doubly-sparse transform that multiplies a fixed, well-conditioned matrix with a data-adaptive sparse component, thereby introducing controllable data adaptivity while preserving fast and stable computation. We devise a structured learning approach incorporating condition number control to balance generalization and approximation accuracy, and develop a novel closed-form projection operator within an inexact proximal framework for efficient optimization. The resulting method achieves state-of-the-art performance in doubly-sparse transform learning, significantly reducing computational cost compared to dense variants, converging faster, and more effectively avoiding poor local minima.

This work addresses the limited sparsity of traditional orthogonal wavelet transforms—despite their multiscale representation and energy conservation properties—which constrains their denoising performance. Departing from the classical wavelet filter bank framework, the paper proposes a composite construction method based on orthogonal wavelet matrices, leveraging matrix products, Kronecker products, and block-diagonal structures to design novel invertible, numerically stable, unitary-like transforms. These transforms preserve perfect reconstruction while substantially enhancing coefficient sparsity, as quantified by the Lorenz curve. Experimental results demonstrate consistently lower mean squared error across diverse datasets, including Donoho–Johnstone test signals, Barbara images, and atmospheric turbulence measurements, indicating superior noise suppression with improved preservation of intrinsic signal structure.

This work addresses the interpretable additive decomposition of one-dimensional signals. We propose the first Transformer-based end-to-end deep learning method that automatically disentangles an input signal into four physically meaningful components: piecewise-constant, smooth (low-frequency), texture (high-frequency), and noise. Unlike conventional approaches relying on hand-crafted mathematical priors (e.g., total variation or sinusoidal models), our method employs a sequence-to-sequence architecture with a multi-branch output head to jointly predict all components, trained exclusively on synthetically generated data. Experiments on in-distribution synthetic signals demonstrate significantly lower reconstruction errors for each component compared to classical methods—including TV-L1 regularization and synchrosqueezing transform—validating the Transformer’s capacity to learn effective signal priors from data. The results establish a new data-driven paradigm for interpretable signal decomposition, highlighting both modeling efficacy and generalization potential.

This work addresses the lack of theoretical characterization in existing kernel methods for machine learning regarding the residual structure and energy stability of multichannel signals in complex systems. The authors propose an analytical framework grounded in operator defect identities, introducing the novel concept of “telescopic energy residuals.” By integrating iterative products with a λₙ-relaxed Kaczmarz scheme, they establish admissibility conditions for residuals and derive prior energy bounds. For the first time, this framework incorporates operator defect theory into kernel methods and kernel principal component analysis (KPCA), rigorously proving explicit convergence of generalized algorithms, a residual energy decomposition theorem, and stability criteria under noise. The approach significantly extends infinite-dimensional Kaczmarz theory to broader applications in machine learning.

This work investigates the impact of matrix concatenation operations on singular value spectra, aiming to bridge a theoretical gap in the structural stability of SVDs under concatenation. Addressing the central question—“How are the singular values of a concatenated matrix determined by those of its constituent submatrices?”—we extend Weyl’s inequality to block-wise concatenation for the first time, establishing a quantitative analytical framework grounded in matrix perturbation theory and norm inequalities. We derive computable upper bounds on singular value deviations and rigorously prove that dominant singular values remain stable when the operator norms of the submatrices are comparable. This result provides theoretical guarantees and principled design guidance for low-rank approximation, robust matrix clustering, and compression algorithms.

This study investigates why singular value decomposition (SVD)-based orthogonalization degrades SO(3) rotation estimation performance during training, despite SVD outperforming the Gram–Schmidt method at inference. By deriving, for the first time, the exact spectral structure of the Jacobian in SVD backpropagation, the authors reveal that small singular values induce severe gradient distortion during training. They further demonstrate that the 6D Gram–Schmidt parameterization suffers from imbalanced gradient flow, providing theoretical support for the superiority of 9D representations. Based on these insights, the work proposes a paradigm that avoids orthogonalization during training and applies SVD only at inference, thereby establishing both theoretical grounding and practical guidance for 9D regression followed by SVD-based projection.

This work proposes an end-to-end, data-driven approach for separating mixed signals corrupted by unknown non-Gaussian interference. Built upon an enhanced SoundStream architecture, the method employs finite scalar quantization (FSQ) to produce discrete representations of target signals and integrates an augmented Transformer decoder to achieve adaptive separation and zero-shot generalization—without requiring prior knowledge or side information about the interference. The model is optimized end-to-end via cross-entropy loss and demonstrates substantial performance gains over existing methods on the MIT RF Challenge dataset. Notably, it achieves a 122-fold reduction in bit error rate for QPSK signals under 5G interference, highlighting its robustness and effectiveness in challenging real-world scenarios.

This work addresses the lack of a systematic connection between classical integral operators and reproducing kernel Hilbert spaces (RKHS) in algebraic signal processing, which has hindered theoretical advances in graph signal processing and learnable filters. The paper establishes, for the first time, an algebraic correspondence between integral operators and RKHS by constructing a unital kernel algebra via the box product of operator symbols, thereby deriving the associated reproducing kernel and characterizing its spectral and algebraic properties. This framework enables exact alignment between graph signal spectral decomposition and RKHS representation, extends naturally to directed graphs, and proves that when the spectral domain of a regularized learning problem is a subset of the signal domain, the optimal filter admits a finite-dimensional RKHS representation—providing a rigorous theoretical foundation for learnable filters in neural architectures based on integral operators.

This work addresses the lack of a unified theoretical foundation for classical and modern signal transforms, which has hindered systematic understanding and automated selection. By leveraging group representation theory, the authors unify a broad class of transforms—including the DFT, DCT, Walsh–Hadamard, Haar wavelets, KLT, spherical harmonics, and fractional Fourier transform—as eigenbases of covariance matrices that are covariant under specific group actions. Central to this framework is the identification of a “matching group” that leaves the signal covariance invariant, combined with the Peter–Weyl theorem and the Algebraic Diversity (AD) formalism. The study further introduces a novel, data-driven polynomial-time algorithm to automatically discover the optimal matching group without expert intervention, enabling automatic transform selection. This approach naturally extends to cutting-edge domains such as massive MIMO systems, graph neural networks, and Transformer attention mechanisms.

This study investigates the intrinsic relationship between the architectural properties of large language models and their compressibility, revealing fundamental limitations of conventional compression approaches. Through over 40 systematic experiments on GPT-2 and Mistral 7B—employing canonical correlation analysis (CCA), R² linearity metrics, KL divergence, and rotation-based quantization—the work identifies five key phenomena: variance does not equate to importance; module behavior depends on input distribution; reconstruction error exhibits a “wall effect”; deeper linear components enhance compressibility; and approximately 30% of token computations are redundant. Building on these insights, the authors propose a block-level functional replacement strategy that achieves 34× compression in the final layer of Mistral 7B with only a 1.71 increase in perplexity. However, multi-block replacement fails due to error accumulation, exposing inherent bottlenecks of static compression and challenging prevailing paradigms.