Positivity sets of hinge functions

2025-03-14

AI Summary

This work investigates the geometric characterization of positivity sets—i.e., the sets of inputs yielding positive outputs—for single-layer ReLU neural networks in ℝ² and ℝᵈ. For ℝ², we establish the first necessary and sufficient condition for a positivity set to be a convex cone and provide a complete classification of all realizable conic positivity sets. For general subsets of ℝᵈ, we derive universal necessary conditions grounded in convex conic geometry and the piecewise-linear structure induced by the ReLU activation (a hinge function). Methodologically, our analysis integrates tools from convex geometry, real algebraic geometry, and the piecewise-linear theory of ReLU networks. Our contributions include: (i) establishing a rigorous theoretical link between neural decision boundaries and classical convex geometry; (ii) constructing nontrivial examples demonstrating the tightness of our conditions; and (iii) proposing necessary conditions that are both theoretically profound and practically verifiable—offering a novel paradigm for understanding the expressive capacity of shallow neural networks.

Technology Category

Machine Learning: Learning with ManifoldsKnowledge Representation and Reasoning: Geometric, Spatial, and Temporal ReasoningSearch and Optimization: Non-convex Optimization

Application Category

Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphsSocial Networks and Social Media: Social media analysis through the lenses of networksEconomics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems

Abstract

In this paper we investigate which subsets of the real plane are realisable as the set of points on which a one-layer ReLU neural network takes a positive value. In the case of cones we give a full characterisation of such sets. Furthermore, we give a necessary condition for any subset of $mathbb R^d$. We give various examples of such one-layer neural networks.

Problem

• Characterize positivity sets of one-layer ReLU networks
• Identify necessary conditions for subsets in R^d
• Provide examples of such neural networks

Innovation

• Characterizes positivity sets for ReLU networks
• Necessary condition for subsets in R^d
• Examples of one-layer neural networks