Fourier Learning Machines: Nonharmonic Fourier-Based Neural Networks for Scientific Machine Learning

2025-09-10

AI Summary

Modeling periodic and aperiodic functions in scientific machine learning remains challenging due to limitations of existing Fourier-based neural networks—particularly their reliance on harmonic constraints and inability to decouple dimensional dependencies. Method: This paper proposes the Fourier Learning Machine (FLM), a novel neural architecture grounded in non-harmonic Fourier series, employing cosine activation units with learnable frequencies, amplitudes, and phases. Under multidimensional separability, FLM enables feedforward spectral basis representation and establishes, for the first time, a bijective mapping between Fourier coefficients and phase-amplitude parameters, supporting equivalent conversion between full spectral bases and phase-shifted forms. Contribution/Results: FLM overcomes harmonic and dimensional coupling restrictions inherent in conventional Fourier networks. Experiments demonstrate that FLM matches or surpasses SIREN and standard MLPs in solving partial differential equations and optimal control problems, validating its efficiency, generalization capability, and expressive power for scientific computing tasks.

Technology Category

• Machine Learning: Deep Neural Architectures and Foundation Models
• Natural Language Processing: Learning & Optimization for NLP
• Search and Optimization: Learning to Search

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Web Mining and Content Analysis: Machine learning and data science for the Web

Abstract

We introduce the Fourier Learning Machine (FLM), a neural network (NN) architecture designed to represent a multidimensional nonharmonic Fourier series. The FLM uses a simple feedforward structure with cosine activation functions to learn the frequencies, amplitudes, and phase shifts of the series as trainable parameters. This design allows the model to create a problem-specific spectral basis adaptable to both periodic and nonperiodic functions. Unlike previous Fourier-inspired NN models, the FLM is the first architecture able to represent a complete, separable Fourier basis in multiple dimensions using a standard Multilayer Perceptron-like architecture. A one-to-one correspondence between the Fourier coefficients and amplitudes and phase-shifts is demonstrated, allowing for the translation between a full, separable basis form and the cosine phase--shifted one. Additionally, we evaluate the performance of FLMs on several scientific computing problems, including benchmark Partial Differential Equations (PDEs) and a family of Optimal Control Problems (OCPs). Computational experiments show that the performance of FLMs is comparable, and often superior, to that of established architectures like SIREN and vanilla feedforward NNs.

Problem

• Designing neural networks for nonharmonic Fourier series representation
• Learning frequencies, amplitudes, phase shifts as trainable parameters
• Creating adaptable spectral basis for periodic and nonperiodic functions

Innovation

• Feedforward network with cosine activation functions
• Learns frequencies, amplitudes, phase shifts as parameters
• Creates adaptable spectral basis for periodic and nonperiodic functions