A Bayesian Vertical Federated Learning Framework for Multivariate Reduced-Rank High-Dimensional Regression

2026-09-18

AI Summary

本文提出了一种名为BayesVFLReg的贝叶斯垂直联邦学习框架，用于解决多变量高维降秩回归问题，通过共享随机草图矩阵保护数据隐私同时实现精确系数估计。

Abstract

Federated learning (FL) has emerged as a leading privacy-preserving framework for collaborative machine learning across decentralized environments. While considerable progress has been made in horizontal federated learning (HFL), where data with common features is distributed across sites, vertical federated learning (VFL), where sites share observations across distinct feature sets, remains less explored. Advancing Bayesian high-dimensional multivariate reduced-rank regression methods for VFL poses unique challenges: (a) stringent privacy regulations preventing local site data sharing, and (b) fitting local regressions overlooks essential modeling aspects like inter-variable correlations. In contrast HFL allows each site to fit a comparable model independently. We present a novel Bayesian VFL framework for multivariate high-dimensional reduced-rank regression, termed BayesVFLReg, which enables precise coefficient estimation while safeguarding both feature and response privacy. Participating sites use a shared random sketching matrix to compress local variables into privacy-preserving sketches. A central server collects these sketches where Bayesian multivariate reduced-rank regression uses Gaussian scale mixture priors. For feature selection, we introduce a single-step post-processing strategy based on mixture-model clustering of the absolute posterior coefficient means to distinguish signal from noise per response variable. BayesVFLReg is computationally scalable for large, high-dimensional datasets and facilitates efficient variable selection. Theoretically, we establish sharp non-asymptotic bounds on the posterior probability that the fitted density falls within a Hellinger ball centered at the true data-generating density. Comparative simulation studies and real-world data analyses show that BayesVFLReg reliably identifies sparse feature effects, even under feature correlation.

Problem

• Vertical Federated Learning
• High-Dimensional Regression
• Privacy Preservation
• Inter-variable Correlations

Innovation

• Bayesian Vertical Federated Learning
• Multivariate Reduced-Rank Regression
• Privacy-Preserving Sketches
• Gaussian Scale Mixture Priors
• Feature Selection