matrix factorization

This paper addresses classification under high-dimensional sparse settings. We propose a two-step discriminant method based on principal component analysis (PCA), grounded in an implicit low-rank factor model and featuring adaptive selection of the number of principal components. We establish, for the first time, a general risk analysis framework for high-dimensional two-step classifiers and rigorously derive the minimax-optimal convergence rate (up to logarithmic factors) for the PCA-based classifier—even when dimensionality far exceeds sample size. Theoretically, the excess risk achieves the optimal rate; simulations demonstrate robustness under model misspecification; and empirical evaluation on three real-world high-dimensional datasets shows significant improvement over state-of-the-art discriminant methods. Key contributions include: (i) a unified theoretical analysis paradigm for two-step classification, (ii) minimax-optimal rate guarantees, (iii) a data-driven, theoretically justified dimension-selection mechanism, and (iv) consistent empirical superiority across diverse high-dimensional benchmarks.

Deep learning training suffers from performance instability and unreliable hyperparameter selection due to high sensitivity to multi-factor interactions. To address this, we propose a risk-aware optimization framework integrating experimental design with Shapley value decomposition. Our method introduces a novel dual-path Shapley estimation: a plug-in estimator based on conditional means ensures interpretability, while a least-squares reconstruction path enables bias correction and cross-factor comparability in heterogeneous parameter spaces; we theoretically derive an upper bound on the optimality gap. The framework integrates standardized Shapley estimation, uncertainty quantification, and lightweight search to efficiently identify robust configurations under budget constraints. Extensive evaluation across multiple datasets and experimental design scenarios demonstrates significant improvements in optimal configuration identification accuracy and rank preservation, effectively reducing decision risk while balancing performance gains and model interpretability.

Traditional topic models impose a “sum-to-one” constraint on parameters, leading to complex optimization landscapes and low computational efficiency. This work proposes integrating nonnegative matrix factorization (NMF) into topic model parameter estimation, eliminating the hard simplex constraint and instead leveraging NMF’s nonnegativity and low-rank structure to implicitly model topic distributions—thereby substantially simplifying the optimization problem. Methodologically, we design an efficient solving framework built upon state-of-the-art NMF algorithms and incorporate a post-processing step to recover interpretable probabilistic parameters; the method is implemented in the R package *fastTopics*. Experiments demonstrate consistent improvements in both accuracy and speed under maximum likelihood estimation and variational inference: superior fit within fixed time budgets, or significantly reduced runtime at equivalent accuracy. To our knowledge, this is the first systematic integration of NMF’s optimization advantages across the entire topic modeling pipeline, offering a new paradigm for high-dimensional text modeling that balances theoretical simplicity with computational scalability.

This work addresses the multi-level low-rank (MLR) matrix approximation problem under the Frobenius norm, tackling three core challenges: hierarchical structural partitioning (row/column stratification), rank allocation (optimizing individual block ranks under a total storage budget), and joint factor fitting. We propose the first end-to-end joint optimization framework for MLR matrices, unifying structural design, rank assignment, and factor learning within a single model. Our approach employs hierarchical block-diagonal parameterization, alternating optimization, and a constrained rank allocation algorithm to achieve coordinated optimization. The resulting approximation preserves matrix-vector multiplication complexity at O(n). Empirical evaluation on multiple benchmark datasets shows that our method reduces approximation error by 35% on average compared to single-level low-rank baselines, significantly improving both accuracy and storage efficiency. The implementation is publicly available.

Bayesian inference for high-dimensional covariance matrices is computationally prohibitive with conventional MCMC methods (e.g., Gibbs sampling), limiting scalability. Method: We propose FABLE, an efficient pseudo-posterior construction framework that bypasses MCMC entirely. Leveraging the “blessing of dimensionality”—a newly identified phenomenon wherein spectral structure stabilizes in high dimensions—we integrate singular value decomposition (SVD) with joint conjugate priors to construct theoretically justified, high-accuracy pseudo-posteriors. FABLE models low-rank structure via factor analysis, combines SVD-based dimension reduction with closed-form conjugate updates, and calibrates Bayesian credible intervals. Contribution/Results: We establish Wasserstein distance convergence guarantees for the pseudo-posterior. Empirical evaluation on simulated data and real gene expression datasets shows estimation accuracy comparable to MCMC, with 10–100× speedup. FABLE thus achieves both statistical reliability and computational scalability for high-dimensional covariance inference.

This work addresses the limitations of existing latent factor models in handling high-dimensional incomplete (HDI) data, which often yield biased and insufficient representations due to their sole reliance on gradient descent optimization. To overcome this issue, the authors propose a novel heterogeneous ensemble approach that uniquely integrates differential evolution and gradient descent to construct two complementary latent factor models. An adaptive weighting mechanism is further introduced to dynamically fuse the strengths of both models during training. This strategy effectively mitigates representation bias and consistently outperforms several state-of-the-art latent factor models across three HDI datasets, demonstrating its superior effectiveness and robustness.

Disentangling shared and study-specific latent factors from high-dimensional data across multiple studies remains a key challenge in cross-disease gene expression analysis. This work proposes a nonlinear multi-study sparse variational autoencoder that, for the first time, integrates a sparse nonlinear factor model into a multi-study framework. By implicitly penalizing the number of latent factors and modeling sparse dependencies between features and latent factors, the method automatically separates shared from study-specific components. Theoretical analysis establishes identifiability of the latent factors, and experiments on platelet gene expression data successfully recover biologically meaningful co-expression modules, demonstrating both the effectiveness and interpretability of the approach.

This work addresses the issue of over-parameterization in clustering skewed random matrices by proposing a family of parsimonious mixture models that integrate the skew-t distribution with bilinear factor analysis. This model family systematically encompasses 256 distinct parameter constraint configurations, substantially reducing model complexity while preserving expressive capacity. An AECM algorithm is employed for efficient parameter estimation. Empirical validation on the MNIST and Olivetti face datasets demonstrates that the proposed approach maintains or even enhances clustering performance despite a significant reduction in the number of parameters, thereby achieving a unified framework for dimensionality reduction, robust modeling, and efficient clustering.

Identifying latent factors that are stable and transferable across distributionally heterogeneous environments is a key challenge for robust cross-environment prediction. This work proposes ATLAS, the first method to strictly disentangle invariant from environment-specific factors via structural conditioning in heterogeneous settings with partially available auxiliary labels, while unifying treatment of both labeled and unlabeled scenarios for transfer prediction. ATLAS integrates invariance-guided latent alignment, exploitation of auxiliary supervision signals, and low-dimensional latent factor regression, and comes with non-asymptotic theoretical guarantees on prediction error. Experiments demonstrate that ATLAS nearly perfectly recovers the true invariant factors and achieves oracle-level regression performance by enabling full latent signal transfer to novel environments.

This study addresses the inherent non-identifiability of latent variables in factor models—manifested as non-uniqueness and distributional shifts—by systematically elucidating their nature in linear factor models and their implications for representation learning, drawing on an interdisciplinary perspective spanning psychometrics, statistics, and artificial intelligence. It establishes a theoretical connection between this identifiability issue and posterior collapse in variational autoencoders. By integrating factor analysis, linear autoencoders, and variational inference within a high-dimensional asymptotic framework, the work proves that latent factors become fully identifiable as the observation dimension tends to infinity. Building on this result, the authors propose a nearly distribution-free estimation method for high-dimensional settings, effectively bridging the theoretical gap between classical factor analysis and modern deep generative models, particularly well-suited for representation learning with ultra-high-dimensional data.