surrogate modeling

This work addresses Bayesian optimal experimental design under computationally expensive models with limited design evaluations. It proposes an adaptive sequential elimination algorithm that significantly reduces the variance and computational cost of nested Monte Carlo estimators by reusing parameter samples, employing common random numbers, and applying Rao–Blackwellization. A bootstrap-based probabilistic comparison mechanism is integrated to iteratively eliminate inferior designs. The method achieves high reliability while drastically reducing the number of model evaluations, making it well-suited for large-scale engineering applications where computational efficiency and decision accuracy must be carefully balanced.

This work addresses the high cost of small-scale pilot experiments required to fit scaling laws in large-scale model training. Framing the problem as a budget-aware sequential experimental design task, the authors propose an uncertainty-aware active selection strategy that dynamically chooses the most informative experiments from a heterogeneous-cost pool for extrapolation to the target regime. By integrating sequential experimental design, uncertainty quantification, and active learning, the method achieves fitting accuracy comparable to that of exhaustive experimentation using only approximately 10% of the total training budget across diverse scaling law tasks, substantially outperforming conventional experimental design baselines.

Bayesian parameter inference for complex stochastic simulators suffers from prohibitive computational cost due to intractable likelihoods—especially in high-dimensional parameter spaces or when simulator outputs are information-poor. To address this, we propose OptiBayes: a novel framework that reformulates stochastic simulation as a differentiable deterministic optimization problem. By leveraging gradient-based optimization, OptiBayes rapidly identifies high-posterior-density regions while substantially suppressing wasteful simulations in low-probability areas. Built upon an optimization–Monte Carlo hybrid paradigm and implemented end-to-end with JAX, it enables full vectorization and differentiability throughout the inference pipeline. Experiments demonstrate that OptiBayes matches or exceeds state-of-the-art methods in estimation accuracy while reducing runtime by one to two orders of magnitude. It further exhibits strong scalability, robustness to noise and model misspecification, and exceptional adaptability to weakly informative simulator outputs.

This paper challenges the unverified implicit assumption in the predict-then-optimize paradigm that “higher prediction accuracy necessarily yields better downstream decisions,” particularly in multiclass classification settings. Method: We propose a controllable, interpretable multiclass prediction simulation framework that explicitly models error types and distributions, enabling systematic analysis of how classification errors affect decision quality in constrained optimization. Contribution/Results: Experiments on job scheduling and other combinatorial optimization tasks reveal a nonlinear relationship between prediction error and decision performance: improving prediction accuracy does not guarantee improved solution quality—and can even degrade decisions when error patterns shift. Our findings question the conventional coupling logic between prediction and optimization, providing theoretical foundations and practical guidance for designing, evaluating, and calibrating classifiers specifically tailored to decision objectives.

Conventional metamodeling is constrained by pre-specified single-output statistics (e.g., mean), limiting its applicability in real-time decision-making where arbitrary statistics must be computed on demand. Method: We propose a generative metamodeling paradigm—designed to serve as a fast surrogate for simulators—that efficiently generates random samples approximating the true conditional distribution given any input. To this end, we formally define generative metamodeling and introduce Quantile Regression-based Generative Metamodeling (QRGMM), a novel algorithm grounded in quantile regression. We provide theoretical guarantees on its conditional distribution convergence and establish its optimal convergence rate. Results: Experiments across diverse real-time decision tasks demonstrate that QRGMM significantly outperforms existing generative models: it achieves 100×–1000× faster inference while preserving high distributional fidelity, thereby overcoming the flexibility bottleneck inherent in traditional single-statistic surrogates.

This work addresses the inefficiency of surrogate models in optimization and simulation tasks that either rely on expensive high-quality labels or encounter complex optimization landscapes. To overcome these challenges, the authors propose a three-stage learning framework: first, supervised pretraining with abundant yet imperfect low-cost labels; second, self-supervised fine-tuning to refine model representations; and third, integration of feasibility constraints to ensure valid solutions. Theoretical analysis demonstrates that only a small number of low-precision labels are sufficient to guide the model into the basin of attraction of the optimal solution, substantially reducing both data and computational costs. Empirical evaluations across nonconvex constrained optimization, power grid scheduling, and rigid dynamical systems show faster convergence, higher accuracy, and improved feasibility, achieving up to a 59-fold reduction in total offline cost.

Traditional conformal prediction (CP) provides only marginal coverage guarantees under small-sample calibration, exhibiting high variance in coverage distribution and frequent violations below the nominal level—thereby undermining reliability in uncertainty quantification. To address this, we propose a novel conformal prediction framework that, for the first time, delivers probabilistic coverage guarantees for individual predictors—e.g., $ mathbb{P}( ext{Coverage} geq 1-alpha) geq 1-delta $—overcoming the fundamental limitation of marginal guarantees. This guarantee holds rigorously even with limited calibration data and asymptotically recovers classical CP guarantees under large samples. Our method leverages nonparametric concentration inequalities, requires no assumptions on error distributions, and integrates seamlessly with mainstream CP libraries. Experiments demonstrate substantial improvements in coverage stability and safety under low-data regimes, providing verifiable statistical guarantees for uncertainty quantification in resource-constrained settings.

Efficient hyperparameter optimization for scale/precision parameters in stochastic models remains challenging under noisy evaluations. Method: This paper proposes a novel Bayesian optimization framework featuring a statistical surrogate model that enables closed-form analytical expressions of the expected acquisition function. Crucially, it derives, for the first time, a closed-form solution for the stochastic acquisition function optimizer—eliminating the need for Monte Carlo sampling. Contribution/Results: The method substantially reduces computational overhead in noisy environments. Evaluated on two computational engineering numerical experiments, it achieves up to a 40× improvement in iteration efficiency, while simultaneously reducing data requirements and total computational cost by approximately 40×, thereby significantly alleviating resource bottlenecks in hyperparameter tuning.

This work addresses uncertainty propagation in multi-fidelity trajectory simulation under constrained computational budgets and without prior allocation of hyperparameter tuning overhead. We propose an online automated hyperparameter optimization framework that exploits correlations and cost disparities between high- and low-fidelity models, dynamically co-optimizing low-fidelity hyperparameters during a single simulation run to adapt configurations in real time. Compared to manual tuning, our method significantly reduces estimation variance and approaches the optimal performance bound under low budgets; in realistic entry-descent-landing mission validation, it improves estimation accuracy by up to 32%, while converging to the known optimal parameter performance boundary under higher budgets. The core innovation lies in embedding hyperparameter optimization directly into the uncertainty propagation pipeline, enabling a tightly coupled, adaptive “simulate-optimize-propagate” mechanism.

This work addresses the computational challenges posed by traditional discrete model selection criteria—such as AIC and BIC—whose objective functions are discontinuous due to their dependence on the number of parameters, rendering them difficult to optimize. To overcome this limitation, the authors propose a provably convergent sequence of smooth surrogate functions that reformulate these criteria as continuous optimization problems. By integrating this approach with explicit over-parameterized modeling, the method enables joint optimization of model selection and parameter inference. This framework circumvents the computational bottlenecks inherent in conventional discrete optimization strategies, significantly reducing computational complexity while achieving superior performance in clustering tasks.