JA-SIREN: Deterministic Initialization for Sinusoidal Networks via Spectral Matching

2026-06-04

AI Summary

This work addresses the instability and irreproducibility of sinusoidal networks caused by random initialization by proposing a deterministic initialization method grounded in spectral analysis. By integrating the Discrete Sine Transform (DST) with the Jacobi–Anger expansion, the authors derive closed-form initial weights for two-layer sinusoidal MLPs that precisely match the spectral response of the target signal at initialization. This approach uniquely combines spectral matching with the Jacobi–Anger expansion to achieve zero-variance initialization without reliance on random seeds or hyperparameter tuning. Evaluated on the Kodak dataset, the method achieves an average PSNR of 67.18 dB—surpassing the best baseline by 21.30 dB—with no performance variation across runs, thereby significantly enhancing the stability and reproducibility of implicit neural representations.

Abstract

Existing implicit neural representation (INR) approaches suffer from stochastic initialization that does not guarantee consistent or high-quality performance across runs, with variations reaching more than 2.5 dB (78%) in image regression. This variation is problematic for scientific computing and simulation, where result reproducibility is crucial. To address this problem, we present Jacobi-Anger Sinusoidal Representation Network (JA-SIREN), a deterministic initialization scheme for sinusoidal networks grounded in classical spectral analysis. By computing the Discrete Sine Transform (DST) of the target signal and leveraging the Jacobi-Anger expansion, we derive closed-form weights for a two-layer sinusoidal MLP that analytically match the network's initial spectral response to the target signal, requiring no random seed or additional hyperparameter tuning. On the Kodak dataset, JA-SIREN achieves a mean PSNR of 67.18 dB, a 21.30 dB improvement over the best baseline. This is achieved with zero run-to-run variance, confirming that spectrally-informed initialization is a more effective and reproducible alternative to stochastic initialization for sinusoidal INRs.

Problem

• implicit neural representation
• stochastic initialization
• reproducibility
• sinusoidal networks
• spectral matching

Innovation

• deterministic initialization
• sinusoidal networks
• spectral matching
• Jacobi-Anger expansion
• implicit neural representation