🤖 AI Summary
This work proposes GAReg, a unified genetic algorithm framework designed to address high-dimensional discrete regression optimization problems where traditional gradient-based methods fail due to non-differentiable, non-convex objective functions or the presence of numerous local optima. GAReg employs compact chromosome encoding, constraint-preserving crossover and mutation operators, a steady-state population update mechanism, and supports island-model parallelization. This approach effectively overcomes the computational bottlenecks of enumeration and dynamic programming in high-dimensional spaces. Experimental results demonstrate that GAReg efficiently explores the model space in scenarios where standard methods are inapplicable, yielding near-optimal solutions and significantly outperforming conventional approaches such as exhaustive search and integer programming.
📝 Abstract
Many statistical problems involve optimization over a discrete parameter space having an unknown dimension. In such settings, gradient-based methods often fail due to the non-differentiability of the objective function or a non-convex or massive search space with an objective function having many local maxima/minima. This paper presents GAReg, a unified genetic algorithm package that handles discrete optimization regression problems, which works well when standard algorithms are unjustified. GAReg provides a compact chromosome representation supporting optimal knot placement for regression splines, best-subset regression variable selection, and related problems. The package allows for uniform initialization, constraint-preserving crossover and mutation, steady-state replacement, and an optional island-model parallelization. GAReg efficiently searches high-dimensional model spaces, providing near-optimal solutions in settings where exhaustive enumeration or integer or dynamic programming approaches are infeasible.