🤖 AI Summary
This work addresses the computational challenges posed by traditional discrete model selection criteria—such as AIC and BIC—whose objective functions are discontinuous due to their dependence on the number of parameters, rendering them difficult to optimize. To overcome this limitation, the authors propose a provably convergent sequence of smooth surrogate functions that reformulate these criteria as continuous optimization problems. By integrating this approach with explicit over-parameterized modeling, the method enables joint optimization of model selection and parameter inference. This framework circumvents the computational bottlenecks inherent in conventional discrete optimization strategies, significantly reducing computational complexity while achieving superior performance in clustering tasks.
📝 Abstract
Models with fewer parameters are often easier to interpret and more robust. Parsimony can be achieved through optimizing objectives like the AIC or BIC, which are functions of the the number of free parameters in the model. Optimizing this discrete objective is a challenge, often relying on discrete optimization. We construct smooth functions with optima that reach the same optima of these objectives but permit continuous rather than discrete optimization, relieving some selection burden. Proofs of convergence are provided and a novel method of clustering through explicit overparamterization shows promising results.