Linear Regression Using Hilbert-Space-Valued Covariates with Unknown Reproducing Kernel

2025-04-23

AI Summary

This paper addresses parameter estimation and asymptotic inference for linear regression with Hilbert space-valued covariates—such as two-dimensional brain images—under an unknown reproducing kernel. To overcome this challenge, we propose a kernel-free estimation method based on principal component analysis (PCA): first performing kernel-free PCA to reduce the dimensionality of the covariates, then fitting linear regression in the low-dimensional projected space. We establish the $sqrt{n}$-consistency and asymptotic normality of the resulting regression coefficient estimator. This work is the first to systematically incorporate Hilbert space covariates with unknown reproducing kernels into the linear regression framework, circumventing explicit RKHS kernel estimation and thereby substantially improving computational efficiency and robustness. Simulation studies and real brain imaging data analyses confirm the consistency, predictive accuracy, and large-sample statistical validity of the proposed estimator.

Technology Category

• Machine Learning: Kernel Methods
• Computer Vision: Learning & Optimization for CV
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

We present a new method of linear regression based on principal components using Hilbert-space-valued covariates with unknown reproducing kernels. We develop a computationally efficient approach to estimation and derive asymptotic theory for the regression parameter estimates under mild assumptions. We demonstrate the approach in simulation studies as well as in data analysis using two-dimensional brain images as predictors.

Problem

• Linear regression with unknown reproducing kernel covariates
• Efficient estimation via principal components method
• Application to brain image predictors analysis

Innovation

• Hilbert-space-valued covariates regression
• Principal components with unknown kernels
• Efficient estimation and asymptotic theory