design training objectives

This work addresses the challenge in aerodynamic inverse design, where high-dimensional geometry and computationally expensive simulations hinder the simultaneous optimization of performance and geometric plausibility. To overcome this, the authors propose a unified framework that integrates optimal design points with design distributions by combining optimization and guided generative modeling. Key innovations include a novel loss function for cost predictor training, a density gradient-based optimization strategy, and an efficient approximate conditional covariance estimation algorithm that enables a guidance generation framework without additional training. The approach is implemented with OpenFOAM simulations and offline reinforcement learning, and validated through 3D-printed wind tunnel experiments. It demonstrates significant performance improvements on both 2D control tasks and high-fidelity 3D benchmarks for automotive and aerospace applications, showcasing both effectiveness and practicality.

This work addresses the inefficiency of global optimization when standard neural network surrogates are embedded into mixed-integer linear programs (MILPs), a challenge stemming from the lack of control over their structural properties. The authors propose a novel differentiable regularizer that, for the first time, approximates the full gradient of the LP relaxation gap with respect to network parameters, enabling direct optimization of key structural attributes such as big-M constants, the number of unstable neurons, and the LP relaxation gap itself. Built upon ReLU networks and MILP formulations, the method leverages gradients from LP dual variables and requires no custom automatic differentiation. Experiments demonstrate up to four orders of magnitude reduction in MILP solve time on nonconvex benchmark functions and two-stage stochastic programming problems, all while preserving predictive accuracy.

Offline model-based optimization (MBO) faces a fundamental challenge: regression models trained on static datasets suffer from out-of-distribution errors, leading to overestimation of suboptimal designs and misguiding the optimization process. This work observes that MBO’s core objective is to **identify promising design rankings**, not to predict absolute performance scores accurately. Accordingly, we propose the first integration of **Learning to Rank (LTR)** into offline MBO. Instead of minimizing mean squared error, our method employs pairwise or listwise ranking losses within an offline reinforcement learning framework to explicitly model relative design preferences. We further derive a theoretical upper bound on the generalization error of ranking loss in this setting. Evaluated across diverse benchmark tasks, our approach consistently outperforms 20 state-of-the-art methods, achieving superior robustness and higher-quality optimal solutions.

Learned optimizers (L2Os) suffer from poor out-of-distribution generalization, limiting their applicability beyond the training data distribution. Method: This paper proposes a novel paradigm integrating classical optimization priors with data-driven modeling. It systematically incorporates fundamental optimization principles—specifically scale invariance and affine covariance—into the architecture design. We introduce a parameterized quasi-Newton update module explicitly constrained to preserve BFGS structure, and jointly optimize it via end-to-end training that unifies optimization-theoretic modeling, neural network architecture design, and meta-learning. Contribution/Results: The resulting enhanced BFGS algorithm significantly outperforms both standard L2Os and conventional solvers on unseen problem classes, dimensions, and condition numbers. It achieves over 40% improvement in cross-distribution generalization performance, establishing a new pathway toward more transferable and robust learned optimizers.

In offline optimization, surrogate models suffer from poor calibration in out-of-distribution regions; existing conditional methods exhibit weak generalization and strong model dependency. This paper proposes a model-agnostic gradient norm regularization that explicitly constrains the local sharpness of surrogate models during training. We are the first to extend sharpness-based generalization theory—from prediction loss to the gradient level—establishing a theoretical bound linking training-set gradient sharpness to worst-case gradient sharpness on unseen data. The proposed regularization is architecture-agnostic and seamlessly integrates into arbitrary surrogate models (e.g., Gaussian processes, neural networks) without structural modification. Empirical evaluation on multi-objective black-box optimization tasks demonstrates an average performance improvement of 9.6%, with significant gains in generalization and robustness. The implementation is publicly available.

This work proposes a novel framework integrating transfer learning with constrained Bayesian optimization to address key challenges in aircraft design, including cold-start conditions, heterogeneous variables, and complex constraints. By constructing an ensemble of surrogate models driven by metadata, and incorporating partial least squares dimensionality reduction alongside tailored strategies for handling heterogeneous variables, the approach significantly enhances prediction accuracy for both the objective function and constraint evaluations. The method accelerates convergence notably during early optimization stages while maintaining a favorable balance between modeling efficiency and optimization performance, thereby offering an effective solution pathway for high-dimensional, heterogeneous engineering design problems.

This work addresses the lack of systematic methodologies in model optimization, which often relies on heuristic choices and struggles to accommodate diverse deployment constraints. It formalizes model compression and acceleration as a constraint-aware multi-objective engineering decision problem, establishing a unified and actionable framework grounded in five key dimensions: data availability, latency, memory footprint, accuracy tolerance, and retraining budget. By integrating techniques such as quantization, pruning, knowledge distillation, parameter-efficient fine-tuning (PEFT), and inference optimization, the study proposes tailored optimization pipelines for four representative industrial scenarios, delivering a reproducible and quantifiable guide for technology selection.

This work addresses the challenge of selecting effective fine-tuning strategies for encoder-decoder pre-trained language models in generation and question-answering tasks. It proposes the Match Task to Objective (MTO) framework, which establishes the first systematic alignment mechanism between downstream tasks and pre-training objectives. MTO automatically constructs training data and prompt templates that are consistent with the original pre-training objective and extends this alignment to soft prompt tuning, thereby enabling precise task–objective matching. Experimental results demonstrate that MTO achieves over 120% performance improvement under few-shot settings compared to existing methods, significantly outperforms strong baselines in full-data scenarios, and substantially enhances the effectiveness of prompt tuning.

Offline black-box optimization often struggles to effectively discover optimal designs—such as molecules or materials—due to scarce or low-quality data. To address this challenge, this work proposes OptBias, a novel framework that, for the first time, integrates meta-learning with synthetic tasks generated by Gaussian processes to explicitly model and transfer optimization preferences. The approach further adapts a surrogate model to the target small dataset through fine-tuning. OptBias is unified in its applicability to both continuous and discrete design spaces and substantially mitigates the poor ranking performance of surrogate models under limited data. Empirical results demonstrate that OptBias consistently outperforms state-of-the-art methods across multiple offline optimization benchmarks, particularly excelling in low-data regimes, and exhibits strong robustness and practical utility.