Schauder Bases for $C[0, 1]$ Using ReLU, Softplus and Two Sigmoidal Functions

2025-06-09

AI Summary

Prior work on neural activation functions has primarily focused on universal approximation, without establishing their capacity to form Schauder bases in classical function spaces. Method: This paper rigorously constructs four unconditional Schauder bases for the Banach space $C[0,1]$—the space of continuous real-valued functions on $[0,1]$—using the ReLU, Softplus, and two sigmoidal variants, via functional-analytic arguments and piecewise smooth approximation techniques. Contribution/Results: It is the first work to prove that each of these four widely used activation functions generates a Schauder basis for $C[0,1]$, thereby providing a rigorous functional-analytic foundation for corresponding neural networks. Specifically, any continuous function on $[0,1]$ admits a unique, norm-convergent series expansion in terms of the associated basis functions. This result extends the theoretical expressive power of activation functions beyond mere approximation capability and establishes, for the first time, functional-space completeness guarantees for networks built upon them—significantly advancing their foundational role in approximation theory and representation learning.

Technology Category

Machine Learning: Other Foundations of Machine LearningReasoning under Uncertainty: Other Foundations of Reasoning under UncertaintyComputer Vision: Other Foundations of Computer Vision

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsSearch and Retrieval-Augmented AI: Web query analysis, representation and understandingWeb Mining and Content Analysis: Large pretrained models with web data

Abstract

We construct four Schauder bases for the space $C[0,1]$, one using ReLU functions, another using Softplus functions, and two more using sigmoidal versions of the ReLU and Softplus functions. This establishes the existence of a basis using these functions for the first time, and improves on the universal approximation property associated with them.

Problem

• Construct Schauder bases for C[0,1] using ReLU
• Build bases with Softplus and sigmoidal functions
• Establish first existence of such bases

Innovation

• Schauder bases with ReLU functions
• Schauder bases with Softplus functions
• Schauder bases using sigmoidal variants