Robust Twoblock Simultaneous Dimension Reduction

2026-03-25

AI Summary

This work proposes a robust two-block simultaneous dimension reduction method for multivariate regression in high-dimensional, small-sample settings contaminated by outliers. The approach uniquely enables separate control of model complexity and sparsity for each variable block, integrating robust statistical estimation, simultaneous dimension reduction, and sparse modeling within a unified framework. It accommodates both dense and sparse formulations, effectively extracting essential information from each data block while yielding robust regression coefficient estimates. Extensive simulations and real-data analyses demonstrate that the method consistently achieves superior robustness and estimation efficiency across diverse dimensional configurations and outlier scenarios. The associated algorithm has been made publicly available.

Technology Category

• Machine Learning: Dimensionality Reduction/Feature Selection
• Intelligent Robots: State Estimation
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs

Abstract

This paper introduces robust twoblock (RTB) simultaneous dimension reduction, which is the first statistically robust method to perform simultaneous dimension reduction in two blocks of variables and allows to fine-tune the model complexity in each block individually. The paper proposes both a dense and a sparse version of the new method. Sparse RTB is the first robust estimator that allows to select both model complexity and the degree of sparsity for each block individually. RTB thereby allows to optimally extract and summarize the relevant portion of information in each block of data, also in the presence of outliers. As a corollary, the estimators can be recombined into a single estimate of regression coefficients for multivariate regression that is operable when the number of variables exceeds the number of cases in each block. An extensive simulation study illustrates that the new methods are resistant to different types of outliers, while maintaining estimation efficiency. across a range of dimensionality settings. These findings both hold true for the dense and the sparse method. The methods' performance is further illustrated on two example data sets and a straightforward algorithm is presented and made accessible in an open source repository.

Problem

• robust dimension reduction
• two-block data
• model complexity
• sparsity
• outliers

Innovation

• robust dimension reduction
• twoblock simultaneous analysis
• sparse modeling
• multivariate regression
• outlier resistance