Prior-Informed Neural Network Initialization: A Spectral Approach for Function Parameterizing Architectures

2026-03-17

AI Summary

Function-parameterized neural networks are highly sensitive to initialization, and conventional data-agnostic initialization schemes often fail to capture the structural characteristics of target signals, leading to slow convergence and unstable performance. This work proposes a prior-guided initialization strategy that, for the first time, integrates data-driven spectral priors into both network initialization and architecture design. Specifically, fast Fourier transform (FFT) is employed to extract seasonal priors that inform model depth and initial state, while residual regression is used to parameterize trend components. Without altering the training procedure, the proposed method significantly accelerates convergence, reduces performance variance, and improves computational efficiency across both synthetic and real-world datasets. Notably, it maintains reconstruction accuracy even when using a lower-dimensional encoder, consistently outperforming standard initialization approaches.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Natural Language Processing: Learning & Optimization for NLP
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Neural network architectures designed for function parameterization, such as the Bag-of-Functions (BoF) framework, bridge the gap between the expressivity of deep learning and the interpretability of classical signal processing. However, these models are inherently sensitive to parameter initialization, as traditional data-agnostic schemes fail to capture the structural properties of the target signals, often leading to suboptimal convergence. In this work, we propose a prior-informed design strategy that leverages the intrinsic spectral and temporal structure of the data to guide both network initialization and architectural configuration. A principled methodology is introduced that uses the Fast Fourier Transform to extract dominant seasonal priors, informing model depth and initial states, and a residual-based regression approach to parameterize trend components. Crucially, this structural alignment enables a substantial reduction in encoder dimensionality without compromising reconstruction fidelity. A supporting theoretical analysis provides guidance on trend estimation under finite-sample regimes. Extensive experiments on synthetic and real-world benchmarks demonstrate that embedding data-driven priors significantly accelerates convergence, reduces performance variability across trials, and improves computational efficiency. Overall, the proposed framework enables more compact and interpretable architectures while outperforming standard initialization baselines, without altering the core training procedure.

Problem

• neural network initialization
• function parameterization
• spectral structure
• data priors
• convergence

Innovation

• prior-informed initialization
• spectral analysis
• function parameterization
• seasonal-trend decomposition
• neural network interpretability