A Geometric Measure of Linear Separability for Neural Representations

2026-06-07

AI Summary

This work addresses the limitation of existing neural classifiers that rely on linear readouts and struggle to capture the geometric structure of class representations, particularly under few-shot conditions where unilateral affine separability cannot be properly assessed. The authors propose a directional Linear Separability Metric (LSM) that quantifies the minimal proportion of competing-class samples intruding into an affine half-space containing all samples of a target class. LSM exhibits asymmetry, class-level granularity, target normalization, and invariance under full-rank linear transformations, thereby distinguishing the effects of linear reparameterizations from those of information loss or nonlinear distortions. An efficient penalty-based affine search algorithm is introduced to estimate LSM in high-dimensional feature spaces while preserving the original discrete constraints. Experiments demonstrate that LSM effectively reveals class intrusion phenomena induced by components such as coordinate gating, offering a novel tool for analyzing the geometry of neural representations.

Abstract

Modern neural classifiers commonly rely on linear readouts, yet predictive metrics alone do not characterize the class-wise geometry of the representations on which such readouts operate. We introduce the directional linear separability measure (LSM), a finite-sample diagnostic for one-sided affine separability. For a target class A and a competing set B, LSM searches over affine halfspaces that contain all samples in A and measures the smallest competing-sample intrusion that must remain on the target side, normalized by |A|. The resulting quantity is asymmetric, class-wise, target-normalized, and applicable to finite representations extracted from neural networks. We establish its supporting-hyperplane characterization, relate it to optimal affine classification accuracy, and prove invariance under full-rank linear embeddings. These results separate changes caused by linear reparameterization from those caused by information loss or nonlinear geometric transformations. We also give a penalty-based affine search for estimating class-wise LSM in high-dimensional features, with reported values computed from the original discrete preservation and violation criterion. Finally, we analyze coordinatewise gated nonlinearities as finite-sample geometric operators and empirically use LSM to diagnose class-wise intrusion across common deep-learning components and architectures.

Problem

• linear separability
• neural representations
• affine separability
• class-wise geometry
• finite-sample

Innovation

• linear separability
• neural representations
• affine classification
• geometric measure
• finite-sample analysis