🤖 AI Summary
This work proposes Exponential Family Discriminant Analysis (EFDA), a generalization of classical Linear Discriminant Analysis (LDA) that overcomes its reliance on Gaussian assumptions by accommodating any distribution within the exponential family. Under the assumption that class-conditional densities belong to the same exponential family, EFDA constructs linear decision rules based on sufficient statistics. This approach provides the first unified extension of the LDA framework beyond Gaussianity, guaranteeing asymptotic calibration and statistical efficiency while revealing structural calibration errors arising from model misspecification. Theoretical analysis integrates maximum likelihood estimation, the Cramér–Rao bound, and formal verification in Lean 4. Empirical evaluations across five non-Gaussian simulation settings demonstrate that EFDA matches or exceeds the classification accuracy of LDA, Quadratic Discriminant Analysis (QDA), and logistic regression, reduces expected calibration error by 2–6 times, and is the only method whose mean squared error consistently converges to zero.
📝 Abstract
We introduce Exponential Family Discriminant Analysis (EFDA), a unified generative framework that extends classical Linear Discriminant Analysis (LDA) beyond the Gaussian setting to any member of the exponential family. Under the assumption that each class-conditional density belongs to a common exponential family, EFDA derives closed-form maximum-likelihood estimators for all natural parameters and yields a decision rule that is linear in the sufficient statistic, recovering LDA as a special case and capturing nonlinear decision boundaries in the original feature space. We prove that EFDA is asymptotically calibrated and statistically efficient under correct specification, and we generalise it to $K \geq 2$ classes and multivariate data. Through extensive simulation across five exponential-family distributions (Weibull, Gamma, Exponential, Poisson, Negative Binomial), EFDA matches the classification accuracy of LDA, QDA, and logistic regression while reducing Expected Calibration Error (ECE) by $2$--$6\times$, a gap that is \emph{structural}: it persists for all $n$ and across all class-imbalance levels, because misspecified models remain asymptotically miscalibrated. We further prove and empirically confirm that EFDA's log-odds estimator approaches the Cramér-Rao bound under correct specification, and is the only estimator in our comparison whose mean squared error converges to zero. Complete derivations are provided for nine distributions. Finally, we formally verify all four theoretical propositions in Lean 4, using Aristotle (Harmonic) and OpenGauss (Math, Inc.) as proof generators, with all outputs independently machine-checked by AXLE (Axiom).