Gradient-Free Generation for Hard-Constrained Systems

2024-12-02

AI Summary

In scientific computing, generative models must strictly satisfy hard physical constraints—such as partial differential equations (PDEs)—yet gradient-based optimization is often infeasible due to gradient sparsity or high computational cost. Method: This paper proposes the Extrapolation–Correction–Interpolation (ECI) sampling framework, the first method to inject exact hard constraints into flow matching models *without gradients*, *without fine-tuning*, and *zero-shot*. Leveraging a pre-trained flow matching model, ECI iteratively refines generation trajectories via constraint-driven extrapolation, projection-based correction, and path interpolation—enforcing diverse PDE constraints rigorously while avoiding gradient computation and model adaptation. Contribution/Results: Experiments demonstrate that ECI achieves superior zero-shot generation quality over existing constrained generation baselines. Moreover, on regression tasks, it attains competitive accuracy without fine-tuning. ECI establishes an efficient, general-purpose paradigm for physics-informed generative modeling.

Technology Category

• Machine Learning: Deep Generative Models & Autoencoders
• Search and Optimization: Sampling/Simulation-based Search
• Computer Vision: Diffusion Models for Vision

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Generative models that satisfy hard constraints are critical in many scientific and engineering applications, where physical laws or system requirements must be strictly respected. Many existing constrained generative models, especially those developed for computer vision, rely heavily on gradient information, which is often sparse or computationally expensive in some fields, e.g., partial differential equations (PDEs). In this work, we introduce a novel framework for adapting pre-trained, unconstrained flow-matching models to satisfy constraints exactly in a zero-shot manner without requiring expensive gradient computations or fine-tuning. Our framework, ECI sampling, alternates between extrapolation (E), correction (C), and interpolation (I) stages during each iterative sampling step of flow matching sampling to ensure accurate integration of constraint information while preserving the validity of the generation. We demonstrate the effectiveness of our approach across various PDE systems, showing that ECI-guided generation strictly adheres to physical constraints and accurately captures complex distribution shifts induced by these constraints. Empirical results demonstrate that our framework consistently outperforms baseline approaches in various zero-shot constrained generation tasks and also achieves competitive results in the regression tasks without additional fine-tuning.

Problem

• Develops gradient-free generative models for hard-constrained systems.
• Ensures strict adherence to physical laws without gradient computations.
• Achieves accurate constrained generation in zero-shot settings.

Innovation

• Gradient-free adaptation of pre-trained models
• ECI sampling for constraint integration
• Zero-shot constrained generation without fine-tuning