Selection of functional predictors and smooth coefficient estimation for scalar-on-function regression models

2025-06-21

AI Summary

This paper addresses scalar-on-function regression with high-dimensional functional predictors. We propose SOFIA (Smooth Oracle Functional Iterative Algorithm), a unified framework that simultaneously tackles functional variable selection and smooth coefficient estimation. SOFIA constructs the functional coefficient space within a reproducing kernel Hilbert space (RKHS) and, for the first time, integrates adaptive Lasso with functional subgradient optimization—achieving both variable selection consistency and asymptotically optimal coefficient estimation, i.e., the functional oracle property. Unlike existing approaches, SOFIA maintains accurate variable screening and robust, smooth coefficient estimation under high-dimensional settings. Extensive simulations and an empirical application to GDP growth forecasting demonstrate SOFIA’s superior performance in both selection accuracy and estimation precision.

Technology Category

Machine Learning: Kernel MethodsSearch and Optimization: Non-convex OptimizationReasoning under Uncertainty: Stochastic Optimization

Application Category

Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingGraph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphsSystems and Infrastructure for Web, Mobile and WoT: Experiences and lessons learnt from Web-based algorithms and system deployments

Abstract

In the framework of scalar-on-function regression models, in which several functional variables are employed to predict a scalar response, we propose a methodology for selecting relevant functional predictors while simultaneously providing accurate smooth (or, more generally, regular) estimates of the functional coefficients. We suppose that the functional predictors belong to a real separable Hilbert space, while the functional coefficients belong to a specific subspace of this Hilbert space. Such a subspace can be a Reproducing Kernel Hilbert Space (RKHS) to ensure the desired regularity characteristics, such as smoothness or periodicity, for the coefficient estimates. Our procedure, called SOFIA (Scalar-On-Function Integrated Adaptive Lasso), is based on an adaptive penalized least squares algorithm that leverages functional subgradients to efficiently solve the minimization problem. We demonstrate that the proposed method satisfies the functional oracle property, even when the number of predictors exceeds the sample size. SOFIA's effectiveness in variable selection and coefficient estimation is evaluated through extensive simulation studies and a real-data application to GDP growth prediction.

Problem

• Select relevant functional predictors for scalar-on-function regression models
• Estimate smooth functional coefficients in high-dimensional settings
• Develop efficient algorithm for variable selection and coefficient estimation

Innovation

• Adaptive penalized least squares algorithm
• Functional subgradients for minimization
• Reproducing Kernel Hilbert Space for regularity