Learning Nonlinear Heterogeneity in Physical Kolmogorov-Arnold Networks

2026-01-20

AI Summary

This work addresses the underutilization of intrinsic computational potential in reconfigurable nonlinear devices by proposing direct training of synaptic nonlinearities in physical hardware. For the first time, learnable physical nonlinearities are employed as native computing units to realize a silicon-on-insulator implementation of the Kolmogorov–Arnold Network (KAN) architecture—termed “Synaptic Nonlinear Elements” (SYNE). Operating robustly at room temperature with microampere-level currents and MHz-speed dynamics, the system leverages heterogeneous nonlinear dynamics to achieve substantial gains in both performance and energy efficiency. In tasks including nonlinear function regression, classification, and lithium-ion battery dynamics prediction, SYNE significantly outperforms software-based multilayer perceptrons (MLPs) and linear physical neural networks of comparable scale, using two orders of magnitude fewer parameters and physical devices.

Technology Category

Machine Learning: Hardware-aware MLCognitive Modeling & Cognitive Systems: Neural Spike CodingSearch and Optimization: Learning to Search

Application Category

Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applicationsEconomics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystemsSearch and Retrieval-Augmented AI: Personalized, context-aware and across-device search

Abstract

Physical neural networks typically train linear synaptic weights while treating device nonlinearities as fixed. We show the opposite - by training the synaptic nonlinearity itself, as in Kolmogorov-Arnold Network (KAN) architectures, we yield markedly higher task performance per physical resource and improved performance-parameter scaling than conventional linear weight-based networks, demonstrating ability of KAN topologies to exploit reconfigurable nonlinear physical dynamics. We experimentally realise physical KANs in silicon-on-insulator devices we term'Synaptic Nonlinear Elements'(SYNEs), operating at room temperature, microampere currents, 2 MHz speeds and ~750 fJ per nonlinear operation, with no observed degradation over 10^13 measurements and months-long timescales. We demonstrate nonlinear function regression, classification, and prediction of Li-Ion battery dynamics from noisy real-world multi-sensor data. Physical KANs outperform equivalently-parameterised software multilayer perceptron networks across all tasks, with up to two orders of magnitude fewer parameters, and two orders of magnitude fewer devices than linear weight based physical networks. These results establish learned physical nonlinearity as a hardware-native computational primitive for compact and efficient learning systems, and SYNE devices as effective substrates for heterogenous nonlinear computing.

Problem

• physical neural networks
• nonlinear heterogeneity
• Kolmogorov-Arnold Networks
• synaptic nonlinearity
• hardware-native computation

Innovation

• physical KAN
• synaptic nonlinearity
• nonlinear computing
• hardware-native learning
• SYNE devices