🤖 AI Summary
To address encoder redundancy and spectral bias in Physics-Informed Neural Networks (PINNs), this paper proposes Spectral PINNSformer (S-Pformer). Methodologically, S-Pformer discards the conventional encoder-decoder architecture in favor of a lightweight, encoder-free decoder-only Transformer; incorporates Fourier feature embeddings to explicitly encode multi-scale frequency-domain information, thereby mitigating low-frequency bias; and leverages self-attention to efficiently capture long-range spatiotemporal dependencies. These design choices jointly reduce model parameter count while enhancing multi-scale representation fidelity for complex PDE solutions. Experimental results demonstrate that S-Pformer consistently outperforms existing PINNSformer variants across multiple benchmark PDE tasks and achieves accuracy comparable to—or exceeding—that of standard MLP-based baselines. Overall, S-Pformer achieves a balanced optimization of predictive accuracy and computational efficiency.
📝 Abstract
Physics-Informed Neural Networks (PINNs) are a useful framework for approximating partial differential equation solutions using deep learning methods. In this paper, we propose a principled redesign of the PINNsformer, a Transformer-based PINN architecture. We present the Spectral PINNSformer (S-Pformer), a refinement of encoder-decoder PINNSformers that addresses two key issues; 1. the redundancy (i.e. increased parameter count) of the encoder, and 2. the mitigation of spectral bias. We find that the encoder is unnecessary for capturing spatiotemporal correlations when relying solely on self-attention, thereby reducing parameter count. Further, we integrate Fourier feature embeddings to explicitly mitigate spectral bias, enabling adaptive encoding of multiscale behaviors in the frequency domain. Our model outperforms encoder-decoder PINNSformer architectures across all benchmarks, achieving or outperforming MLP performance while reducing parameter count significantly.