On the Spectral Structure and Objective Equivalence of Orthogonal Multilabel Fisher Discriminants

2026-05-04

AI Summary

This work addresses the lack of theoretical characterization for discriminative subspaces under orthogonality constraints in multi-label linear discriminant analysis, particularly concerning objective equivalence, effective dimensionality, and estimation error. Focusing on multi-label Fisher discriminant analysis with Stiefel orthogonality constraints, we integrate algebraic structure and statistical analysis to reveal the rank properties of the between-class scatter matrix and establish conditions under which four Fisher objectives become equivalent. We further prove that the effective discriminative dimension can exceed the classical \(C-1\) upper bound. Our analysis yields nearly minimax-optimal subspace estimation error bounds—upper bound \(O(k_{\max}\sqrt{d\log d/n}/\text{gap}_r)\) and lower bound \(\Omega(\sigma^2 d/(n\cdot\text{gap}_r))\)—and provides the first distance preservation and robustness guarantees for the multi-label setting. Numerical experiments validate key algebraic identities and the influence of multi-label-specific parameters such as \(k_{\max}\) and \(\kappa(S_t^{\text{ML}})\).

Abstract

We provide a unified theoretical analysis of Linear Discriminant Analysis with simultaneous multilabel scatter matrix formulations and Stiefel orthogonality constraints. Our contributions span both algebraic structure and statistical guarantees. On the algebraic side, we characterize the rank of the multilabel between-class scatter matrix, showing that the effective discriminant dimensionality can strictly exceed the classical single-label bound of $C-1$; we establish a multilabel partition of variance and prove that all four Fisher objectives are equivalent under the $W^\top S_t^{ML} W = I_r$ constraint while characterizing their divergence under the Stiefel constraint; and we prove a two-sided label-distance preservation bound relating projected distances to Hamming distances in label space. On the statistical side, we establish a finite-sample $O(k_{\max}\sqrt{d\log d/n}/gap_r)$ bound on the subspace estimation error under sub-Gaussian noise with a matching $Ω(σ^2 d/(n\,gap_r))$ minimax lower bound, establishing a near-minimax-optimal rate (matching up to logarithmic and $k_{\max}$ factors) for multilabel discriminant subspace estimation. We further provide high-probability distance concentration, robustness guarantees under label interactions, and a regularization analysis preserving the spectral structure when $d \gg n$. All results are verified numerically on synthetic data generated from the linear label-effect model, covering both the algebraic identities and the multilabel-specific quantities ($k_{\max}$, $κ(S_t^{ML})$, $\|Γ/n\|_2$, $Δ_r$) that govern the statistical bounds. The numerical experiments are designed as a sanity check for the theorems rather than as an empirical benchmark; evaluation on real multilabel datasets is left to future work targeting application-oriented venues.

Problem

• Multilabel Fisher Discriminant
• Orthogonal Constraints
• Scatter Matrix
• Subspace Estimation
• Spectral Structure

Innovation

• Multilabel Fisher Discriminant
• Stiefel Orthogonality
• Subspace Estimation Error
• Spectral Structure Preservation
• Near-Minimax Optimality