pared: Model selection using multi-objective optimization

2025-05-27

AI Summary

In statistical modeling, conventional penalized model selection relies on a single criterion (e.g., prediction error), failing to jointly optimize sparsity, interpretability, and smoothness. To address this, we propose a Bayesian multi-objective model selection framework that—novelty—integrates Gaussian process surrogate models into multi-objective hyperparameter tuning, enabling systematic identification of the Pareto-optimal front. The method accommodates mainstream penalized estimators, including elastic net and fused lasso. Implemented in the R package *pared*, it automates Pareto-front discovery and supports interactive visualization via *ggplot2* and *plotly*. Extensive experiments on synthetic and real-world datasets demonstrate that our framework significantly improves the simultaneous optimization of goodness-of-fit, sparsity, and smoothness. By unifying these competing objectives within a coherent probabilistic framework, it establishes a new paradigm for interpretable machine learning.

Technology Category

• Machine Learning: Auto ML and Hyperparameter Tuning
• Search and Optimization: Mixed Discrete/Continuous Search
• Reasoning under Uncertainty: Stochastic Optimization

Application Category

• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Motivation: Model selection is a ubiquitous challenge in statistics. For penalized models, model selection typically entails tuning hyperparameters to maximize a measure of fit or minimize out-of-sample prediction error. However, these criteria fail to reflect other desirable characteristics, such as model sparsity, interpretability, or smoothness. Results: We present the R package pared to enable the use of multi-objective optimization for model selection. Our approach entails the use of Gaussian process-based optimization to efficiently identify solutions that represent desirable trade-offs. Our implementation includes popular models with multiple objectives including the elastic net, fused lasso, fused graphical lasso, and group graphical lasso. Our R package generates interactive graphics that allow the user to identify hyperparameter values that result in fitted models which lie on the Pareto frontier. Availability: We provide the R package pared and vignettes illustrating its application to both simulated and real data at https://github.com/priyamdas2/pared.

Problem

• Model selection lacks consideration for sparsity and interpretability
• Multi-objective optimization improves trade-offs in model selection
• R package enables efficient Pareto frontier visualization

Innovation

• Multi-objective optimization for model selection
• Gaussian process-based optimization for trade-offs
• Interactive graphics for Pareto frontier analysis