adaptive transfer lasso

This study addresses the challenge of efficiently transferring knowledge from a source domain to enhance estimation accuracy and computational efficiency in quantile regression. The authors propose a novel transfer learning approach that integrates a double L1 penalty, wherein an adaptive LASSO penalty is constructed using estimates derived from the source data to facilitate effective knowledge transfer. This method represents the first integration of transfer learning with adaptive LASSO quantile regression, offering consistent estimation, sparse variable selection, and robustness under non-Gaussian error distributions, while substantially reducing computational complexity. Theoretical analysis establishes its convergence rate and asymptotic properties, and both simulation studies and real-data analysis on protein tertiary structure demonstrate superior performance over conventional LASSO in terms of estimation accuracy and computational efficiency.

This paper addresses the lack of theoretical characterizations for the effective degrees of freedom (EDF) of adaptive Lasso and adaptive group Lasso. We propose, for the first time, unbiased EDF estimators under both orthogonal and non-orthogonal designs. Building upon the Stein’s unbiased risk estimation framework and leveraging structural analysis of penalized regression, we rigorously derive explicit closed-form EDF expressions, revealing the coupled influence of regularization parameters, coefficient signs, and least-squares initial estimates on model complexity. The proposed estimators require neither resampling nor approximation, offering both analytical tractability and broad applicability. Empirical evaluation on synthetic and real-world datasets demonstrates that our EDF estimator significantly improves model selection consistency of information criteria (e.g., AIC, BIC) and enhances the accuracy of prediction error estimation. This work provides a critical theoretical and practical tool for assessing and deploying adaptive regularization methods.

This work addresses the hyperparameter selection challenge in transfer learning for high-dimensional sparse regression, focusing on controlling information transfer strength in Lasso-based methods such as Trans-Lasso. Methodologically, we conduct the first sharp asymptotic analysis of such transfer learning via the replica method. Our theoretical analysis reveals an intrinsic simplicity in transfer behavior: omitting one of the two types of transferred information incurs negligible degradation in generalization performance—effectively reducing the critical hyperparameter space from two- to one-dimensional. This insight substantially simplifies hyperparameter tuning. Empirical evaluation on semi-synthetic datasets derived from IMDb and MNIST demonstrates that our strategy achieves near-optimal predictive performance while significantly reducing hyperparameter search overhead. The results provide both interpretable theoretical guidance and a practical, deployable framework for high-dimensional transfer learning.

This paper addresses estimation and inference in high-dimensional instrumental variable (IV) regression, where both covariates and instruments have dimension $p$ that may greatly exceed the sample size $n$. For estimating second-stage coefficients, we propose embedding either the BRIDGE or adaptive LASSO as penalty functions within the two-stage least squares (2SLS) framework. Theoretically, we establish, for the first time under sub-Gaussian errors, model selection consistency and oracle efficiency of both methods in high-dimensional IV settings. BRIDGE relaxes distributional assumptions—its consistency holds even without sub-Gaussianity when $p > n$, yielding weaker theoretical conditions. Adaptive LASSO achieves comparable asymptotic properties with superior computational efficiency. Together, the two methods offer complementary advantages: BRIDGE provides enhanced robustness, while adaptive LASSO delivers practical scalability. Our work thus furnishes a theoretically grounded yet implementable solution for sparse high-dimensional IV regression.

This work addresses the optimal adaptive aggregation of source and target domain samples in transfer learning to minimize the target risk. To overcome the limitation of existing methods—namely, their inability to uniformly handle diverse distribution divergence measures—we propose a unified weak/strong transfer modulus framework. This is the first approach that automatically adapts to multiple divergence classes—including Wasserstein distance and integral probability metrics (IPMs)—and characterizes their statistical limits. By integrating confidence-set reduction, modulus upper-bound derivation, and adaptive weighted estimation, we achieve near-optimal convergence rates even when the transfer modulus is unknown. Theoretical analysis further reveals that, under causal modeling assumptions, the framework yields provable generalization gains beyond standard transfer bounds. Extensive experiments demonstrate significant improvements in cross-domain classification and regression performance.

This work addresses the challenge of efficiently estimating non-pathwise differentiable functionals—such as dose–response curves under continuous exposure—by proposing a novel approach that integrates higher-order highly adaptive lasso (HAL), spline basis projection, and Targeted Maximum Likelihood Estimation (TMLE). The method projects the target functional onto a finite-dimensional space spanned by higher-order spline bases to construct a pathwise differentiable approximation, and embeds LASSO regularization within the TMLE framework to enable fully data-adaptive inference without requiring pre-specified sieves or parametric models. The resulting estimator exhibits pointwise asymptotic normality, with convergence rates determined solely by the dimensionality and smoothness of the target functional, and demonstrates markedly superior performance over conventional HAL plug-in estimators in simulations.

This work addresses the computational and statistical challenges of post-selection inference in high-dimensional quantile regression by proposing the first distributed selective inference framework. The method innovatively integrates a response proxy strategy with randomized Lasso to transform the nonsmooth quantile loss into a penalized least squares problem. By precisely characterizing the selection event via KKT conditions, it enables efficient inference with only three rounds of communication. Under standard regularity conditions, the asymptotic validity of the proposed inference procedure is rigorously established. Extensive simulations and empirical analyses further demonstrate its superior finite-sample performance.

This study addresses the challenge of integrating heterogeneous prior information in high-dimensional generalized linear models, where existing methods struggle to effectively assess and weight priors of varying quality. The authors propose an adaptive multi-prior Lasso approach that, for the first time, enables data-driven, dynamic weighting of prior information within a unified regularization framework. This method automatically reinforces reliable priors while suppressing unreliable ones, all with theoretical guarantees. As demonstrated through simulations and real-world analysis of TCGA breast cancer gene expression data, the proposed technique substantially improves variable selection accuracy, estimation efficiency, and predictive performance.

This work proposes a novel approach to quantile regression under high-dimensional grouped covariates by introducing, for the first time, an adaptive sparse group Lasso penalty that simultaneously incorporates both within-group and between-group sparsity. By integrating adaptive Lasso and adaptive group Lasso penalties, the method effectively captures the dual sparsity structure while preserving the robustness inherent to quantile regression. To enable efficient computation, the authors develop an alternating direction method of multipliers (ADMM) algorithm based on the dual formulation and rigorously establish its global convergence. Extensive experiments on both simulated and real-world datasets demonstrate that the proposed method substantially outperforms existing alternatives, achieving accurate dual sparsity recovery with superior computational efficiency.