From Data to Program: Fast&Direct Generative Program Inference from Empirical Data

2026-09-28

AI Summary

This study addresses the limited generalization of conventional density estimation, which requires fitting models to specific datasets. We propose PRODiGI, a framework that transforms empirical data into executable explicit generative programs via a single forward pass. Efficient inference is achieved through template prediction and non-autoregressive decoding. Furthermore, we introduce program-space fine-tuning—a novel technique that freezes model parameters while optimizing only program parameters—combined with maximum mean discrepancy (MMD) matching to enhance distributional fidelity. The proposed method outperforms existing baselines in density and score mean absolute error while achieving several-fold faster inference. Notably, program-space fine-tuning reduces the MMD of generated distributions by 84%, enabling efficient and accurate zero-shot density estimation.

Abstract

Estimating probability densities from a finite set of samples typically requires dataset-specific model fitting. We introduce PRODiGI, a pretrained data-to-program model that infers an explicit, executable generative program in a single forward pass. Pretrained on synthetic datasets paired with their ground-truth programs, PRODiGI accommodates diverse generative families and data dimensionalities through template prediction and non-autoregressive program parameter decoding. Its inferred programs support direct sampling, density and score evaluation, and inspection independently of the pretrained model. We further introduce program-space fine-tuning, which refines differentiable program parameters by matching generated and empirical samples while keeping model parameters intact. Experiments show that PRODiGI achieves lower average density and score MAE than existing pretrained models, while offering multi-fold speedups over its closest competitors. Program-space fine-tuning further reduces generation MMD by 84%. By turning empirical data into explicit, reusable programs, PRODiGI introduces a new direction for fast, interpretable tabular generative modeling.

Problem

• probability density estimation
• generative program inference
• tabular generative modeling
• data-to-program

Innovation

• Generative Program Inference
• Non-autoregressive Decoding
• Program-space Fine-tuning
• Pretrained Model
• Tabular Generative Modeling