class subspace orthogonalization

Existing backdoor detectors exhibit insufficient sensitivity to target-class attacks and are prone to interference from class-intrinsic features. Method: We propose Class Subspace Orthogonalization (CSO), a novel detection framework that constructs a constrained optimization problem using a small set of clean samples; it orthogonalizes class-specific feature subspaces to suppress intrinsic class separability and explicitly decouple and amplify the statistical signal of backdoor triggers. CSO integrates decision-confidence analysis with subspace regularization to enable fine-grained detection against stealthy triggers, mixed-label attacks, and adaptive attacks. Contribution/Results: Experiments demonstrate that CSO significantly improves detection sensitivity—especially for weak triggers and scenarios where non-target classes are highly distinguishable—while reducing false positive rates by up to 32.7%. It outperforms state-of-the-art methods in robustness across diverse attack settings.

To address weak class discriminability and insufficient feature robustness in machine learning, this paper proposes a Multi-level Orthogonal Subspace (MOS) Karhunen–Loève feature theory within a random tensor space. Training data are modeled as stochastic processes in a Bochner space, and hierarchical KL expansions explicitly decouple dominant class structures from inter-class anomalous signals, enabling class-wise subspace disentanglement and interpretable projection features. This work establishes, for the first time, a MOS feature construction paradigm under the random tensor framework—uniquely integrating statistical modeling rigor with geometric interpretability. Evaluated on the ADNI plasma dataset, the method significantly outperforms gradient boosting, RUS Boost, random forests, and CNNs, achieving substantial gains in classification accuracy. These results validate its robust discriminative capability for high-noise biomedical data.

This paper addresses the challenge of distinguishing in-distribution (ID) from out-of-distribution (OOD) samples in real-world deep learning deployments, with particular focus on both far-OOD and near-OOD scenarios. We propose an OOD detection method grounded in singular value decomposition (SVD) of the classification head’s weight matrix, which orthogonally decomposes deep-layer activations into two subspaces: a “dominant” subspace governing ID prediction and a “residual” subspace exhibiting higher discriminability for far-OOD inputs. By adaptively fusing confidence scores from both subspaces, our approach achieves shift-agnostic robustness without requiring auxiliary training or architectural modifications. Evaluated on standard benchmarks—including CIFAR and ImageNet—our method consistently outperforms state-of-the-art approaches, delivering substantial improvements in both far-OOD and near-OOD detection accuracy.

To address the lack of structural consistency and non-nested nature of representations across dimensions in low-dimensional subspace learning, this paper proposes a nested subspace modeling framework grounded in flag manifolds. The core methodological innovation is the systematic introduction of the “flag trick”—a novel geometric technique that enforces hierarchical, nested subspace sequences via nested orthogonal projection operators, while preserving the intrinsic optimization structure of the Grassmann manifold. This approach provides a geometric generalization of classical linear dimensionality reduction methods—including PCA and LDA—without requiring additional hyperparameters. Experiments demonstrate substantial improvements in cross-dimensional feature reusability and downstream task stability. Theoretical analysis confirms the framework’s mathematical soundness, and empirical evaluation on multiple benchmarks validates both its theoretical completeness and practical efficacy.

In multi-view subspace learning, theoretical guarantees for distinguishing shared and individual signal subspaces from high-dimensional noisy data remain lacking. This paper establishes, for the first time, necessary and sufficient conditions for subspace separability and develops a rigorous theoretical framework based on spectral perturbation analysis of projection matrix products. Integrating rotational bootstrap with random matrix theory, we propose a parameter-free, interpretable method that automatically partitions subspaces into three categories—shared, individual, and noise—without manual tuning. Leveraging principal angle analysis and diagnostic visualization, our approach enhances estimation robustness. Extensive simulations demonstrate substantial improvements in estimation accuracy for both joint and individual subspaces over state-of-the-art methods. On real-world multi-omics colorectal cancer and murine nutritional genomics datasets, downstream classification and prediction performance is significantly enhanced.

This study investigates why singular value decomposition (SVD)-based orthogonalization degrades SO(3) rotation estimation performance during training, despite SVD outperforming the Gram–Schmidt method at inference. By deriving, for the first time, the exact spectral structure of the Jacobian in SVD backpropagation, the authors reveal that small singular values induce severe gradient distortion during training. They further demonstrate that the 6D Gram–Schmidt parameterization suffers from imbalanced gradient flow, providing theoretical support for the superiority of 9D representations. Based on these insights, the work proposes a paradigm that avoids orthogonalization during training and applies SVD only at inference, thereby establishing both theoretical grounding and practical guidance for 9D regression followed by SVD-based projection.

This work challenges the common practice in knowledge distillation of naively matching a teacher model’s absolute feature representations, which overlooks the fact that such representations are only equivalent up to orthogonal transformations and isotropic scaling. From a geometric perspective, the paper proposes a new paradigm centered on representation equivalence classes: the student should instead learn class-invariant structures of the teacher’s representations—such as Gram matrices, centered kernel alignment (CKA), or principal subspaces—or leverage coordinate alignment for effective supervision. This framework unifies feature matching, relational distillation, and grafting approaches, revealing that logit-level matching is ultimately key to capability transfer. Experiments on Qwen2.5 and Llama-3.1 demonstrate that high CKA similarity alone is insufficient for performance recovery, while successful grafting hinges on boundary overlap in the training data coverage, thereby validating the proposed theory.

This work addresses the security vulnerability of deep learning models that are prone to mislabeling unknown-class samples as a target class under data poisoning attacks. To counter this threat, the paper introduces a novel “latent class attack” paradigm and proposes a post-training detection method that operates without access to the original training data. The approach leverages Class Subspace Orthogonalization (CSO) to analyze internal representations of deep networks, enabling the identification of mislabeled unknown-class samples. It further incorporates visualization techniques to reconstruct and validate suspicious inputs. Experimental results demonstrate that the proposed method effectively detects latent class attacks in image classification tasks, achieving high detection accuracy while offering strong interpretability.

Standard boosting methods often suffer from redundancy among base learners due to repeatedly fitting correlated errors. This work proposes SCBoost, a novel framework that reformulates boosting from a geometric perspective. SCBoost introduces Spectral Residual Projection (SRP) to constrain each new learner to the orthogonal complement of the subspace spanned by previous predictions, ensuring it captures only previously unexplained information. Additionally, Covariance-Regularized Weighting (CRW) is employed to optimize ensemble weights, explicitly reducing inter-learner correlation. The approach enables an exact additive decomposition of residual energy and provably enhances the signal-to-noise ratio under isotropic noise assumptions. Empirical evaluations across ten benchmark datasets demonstrate that SCBoost significantly outperforms baseline methods, achieving particularly notable gains in accuracy and F1 score.

This study addresses the limitation of conventional research that reduces neural network interference to geometric overlap while neglecting code statistics and actual interactions. We propose the concept of "effective interference," which integrates feature geometry with coding statistics to distinguish constructive from destructive interference and quantify interaction strength. Based on sparse autoencoders and a local fixed-support assumption, this work reveals that constrained architectures shape interference patterns through four mechanisms, including orthogonalization and bias compensation. Our findings demonstrate that architectural constraints selectively reduce co-activation overlap while preserving beneficial cross-contributions. These results establish that interference inherently depends on usage patterns and network architecture, thereby overcoming the theoretical limitations of purely geometric perspectives.