SLIM: Stochastic Learning and Inference in Overidentified Models

2025-10-23

AI Summary

To address the computational inefficiency and poor scalability of nonlinear Generalized Method of Moments (GMM) in large-scale overidentified models, this paper proposes SLIM: a stochastic learning framework that iteratively estimates parameters using random minibatches of moment conditions—without requiring consistent initial estimators or global convexity assumptions, and supporting both fixed-sample and random-sampling asymptotics. Innovatively, we develop a family of Sargan–Hansen *J*-tests tailored to stochastic learning, and integrate stochastic approximation, minibatch gradient updates, randomized scaling, and debiased plug-in estimation, enabling compatibility with both first- and second-order optimization. In an EASI demand system featuring 576 moment conditions, 380 parameters, and 10⁵ observations, SLIM completes estimation and inference in just 1.4 hours—12× faster than conventional full-sample GMM—while scaling smoothly to million-scale datasets, thereby achieving both statistical rigor and computational scalability.

Technology Category

• Reasoning under Uncertainty: Stochastic Optimization
• Machine Learning: Scalability of ML Systems
• Search and Optimization: Non-convex Optimization

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• User Modeling, Personalization and Recommendation: Practical large-scale studies of user experience
• Security and Privacy: Large-scale security measurements

Abstract

We propose SLIM (Stochastic Learning and Inference in overidentified Models), a scalable stochastic approximation framework for nonlinear GMM. SLIM forms iterative updates from independent mini-batches of moments and their derivatives, producing unbiased directions that ensure almost-sure convergence. It requires neither a consistent initial estimator nor global convexity and accommodates both fixed-sample and random-sampling asymptotics. We further develop an optional second-order refinement and inference procedures based on random scaling and plug-in methods, including plug-in, debiased plug-in, and online versions of the Sargan--Hansen $J$-test tailored to stochastic learning. In Monte Carlo experiments based on a nonlinear EASI demand system with 576 moment conditions, 380 parameters, and $n = 10^5$, SLIM solves the model in under 1.4 hours, whereas full-sample GMM in Stata on a powerful laptop converges only after 18 hours. The debiased plug-in $J$-test delivers satisfactory finite-sample inference, and SLIM scales smoothly to $n = 10^6$.

Problem

• SLIM enables scalable stochastic learning for nonlinear GMM models
• It eliminates the need for consistent initial estimators or global convexity
• The framework provides efficient inference procedures for large-scale problems

Innovation

• Stochastic approximation framework for nonlinear GMM
• Iterative updates from mini-batches ensure convergence
• Second-order refinement and inference procedures developed