Adaptive Regularization Selection via Stability for Multivariate Regression: Theoretical Guarantees and Real-World Validation

Abstract

 High-dimensional multivariate regression requires regularization for stable estimation and variable selection, yet choosing the tuning parameter that yields reproducible and consistent model recovery remains an open challenge. We propose Adaptive Regularization Selection via Stability (ARSS), a two-stage procedure that (i) constructs data-driven adaptive weights for a convex weighted  penalty, and (ii) selects the regularization level by minimizing a subsampling-based instability index computed across a grid of . We develop a unified theoretical framework proving existence and uniqueness of the estimator, finite-sample stability (perturbation/Lipschitz) bounds, and selection consistency: the stability-selected  attains the high-dimensional scaling  and yields  under standard restricted eigenvalue and beta-min conditions. Under additional weight-decay assumptions an oracle property (asymptotic normality on the active set) is established. Empirically, extensive simulations acr, p, sparsity, , sparsity and correlation structures show that ARSS attains higher true positive rates, markedly lower false discovery rates, and substantially improved selection stability compared to LASSO, Adaptive LASSO, Elastic Net, SCAD, and classical stability selection. Real-world validations on biomedical and prognostics datasets (e.g., UCI cardiovascular and NASA C-MAPSS) corroborate the method’s interpretability and robustness. ARSS provides a principled, reproducible route to tuning regularization in high-dimensional multivariate problems, with practical recommendations for subsampling and grid design.