entropy-maximizing generation

Traditional generative models merely imitate data distributions, struggling to recover diversity lost during training and lacking the capacity for creative out-of-distribution generation. This work proposes the Imaginative Generative AI (IGA) framework, which uniquely treats diversity as a core objective in target distribution design. Introducing the concept of an “entropy wall,” IGA establishes a reference-free diversity control mechanism through spectral entropy of kernel covariance operators and von Neumann entropy. The method integrates KL-anchored exponential tilting optimization with a retraining-free inference guidance strategy—IGA Guidance—compatible with DDPM/DDIM, enabling simultaneous fidelity to the reference distribution and precise diversity modulation. Experiments demonstrate that IGA can restore diversity within the entropy wall and achieve controllable spectral extrapolation beyond it, thereby validating a continuous and controllable transition from imitation to imagination in generative modeling.

This work addresses a pervasive bias in deep generative models—their systematic underestimation of data diversity after training. For the first time, the study identifies the entropy-based origin of this bias through the lens of finite-sample statistical properties. To quantify the diversity gap between generated samples and real data, the authors employ reference-free diversity metrics, including Vendi and RKE scores. Building on this insight, they propose a diversity-aware regularization and guidance strategy. Extensive experiments across multiple benchmark datasets demonstrate that the proposed approach significantly enhances the diversity of generated samples and effectively mitigates the issue of diversity collapse.

Existing evaluation metrics for generative models suffer from difficulties in jointly quantifying fidelity and diversity, strong domain dependence, and insufficient sensitivity to quality variations. Method: This paper proposes a unified, information-theoretic three-dimensional evaluation framework. It systematically integrates k-nearest-neighbor (kNN) density estimation into entropy and cross-entropy theory, yielding three orthogonal metrics—PCE (fidelity), RCE (inter-class diversity), and RE (intra-class diversity)—enabling decoupled sample-level and mode-level analysis for the first time. Results: Experiments demonstrate high sensitivity of each metric to its targeted quality dimension; expose implicit biases in mainstream metrics (e.g., Precision/Recall) regarding mode coverage and real-sample alignment; and confirm strong cross-domain generalization across diverse generative tasks. The framework establishes a theoretically consistent, interpretable, and domain-agnostic paradigm for universal generative model evaluation.

In human-AI collaborative decision-making, tension arises between algorithmic recommendations and human judgment, particularly under data overload, where both observable quantitative objectives and unobservable qualitative preferences (e.g., political feasibility, community acceptability) must be jointly considered. Method: We propose a generative curation framework that dynamically balances optimality and diversity via a dual-path architecture: differentiable sampling and sequential multi-objective optimization. It integrates Gaussian process modeling of latent qualitative utility with a novel diversity metric. Contribution/Results: The framework ensures controllable recommendation set size, near-optimality, and high coverage. Evaluated on policy simulation and operations management tasks, it achieves a 37% improvement in candidate set coverage and a 29% increase in user adoption rate, significantly enhancing decision efficiency and robustness.

To address the prevalent reward collapse problem in diffusion model fine-tuning, this paper proposes an entropy-regularized stochastic control framework and— for the first time—rigorously extends it to general *f*-divergence regularization. Methodologically, we formulate a continuous-time stochastic control model, integrating Itô calculus with variational inference to derive a computationally tractable and provably convergent optimal control policy. Theoretically, we establish that the proposed regularization effectively mitigates reward collapse; empirically, it significantly improves both sample quality and diversity. Key contributions include: (1) the first rigorous stochastic control analysis framework specifically designed for diffusion model fine-tuning; (2) a unified generalization of entropy regularization to arbitrary *f*-divergences, substantially enhancing methodological generality and robustness; and (3) a practical fine-tuning paradigm implementable under multiple divergence metrics.

This work addresses the limitations of existing tools and creative bottlenecks faced by composers and sound designers in timbral exploration by proposing an evolutionary generative framework that integrates Quality-Diversity (QD) algorithms with supervised discriminative models. The approach employs multi-band specialized Compositional Pattern Producing Networks (CPPNs) to reduce architectural complexity while preserving performance, and leverages the MAP-Elites algorithm to efficiently search an extended behavioral space spanning multiple duration dimensions, thereby uncovering mechanisms for cross-contextual target switching. The system autonomously generates synthetic sounds exhibiting both diversity and novelty across temporal and contextual dimensions, with its creative potential validated through an online explorer and experimental musical applications.

Existing quality-diversity (QD) evolutionary algorithms for digital sound synthesis rely on handcrafted features or supervised classifiers, introducing expert bias and constraining exploratory breadth. To address this, we propose an unsupervised, dynamically reconstructed behavioral space framework: acoustic features are automatically reduced in dimensionality via PCA and autoencoders; the low-dimensional embeddings serve as behavior descriptors in MAP-Elites, and the dimensionality-reduction models are periodically retrained to adapt to evolutionary dynamics. This eliminates reliance on manual priors, enabling online evolution and adaptive partitioning of behavioral representations. Experiments demonstrate that our framework significantly enhances sound diversity—particularly with PCA-based compression—effectively mitigates evolutionary stagnation, and supports large-scale, fully automated exploration of parameter spaces. It establishes a scalable, unsupervised QD optimization paradigm for generative audio synthesis.

This study investigates the evolutionary dynamics of generative models trained iteratively on synthetic data contaminated with real data, aiming to mitigate model collapse induced by data pollution. Through statistical modeling, mixture distribution analysis, and theoretical analysis of iterative training dynamics—complemented by theoretical derivations and simulations based on next-token prediction language models—the work demonstrates that model collapse can be effectively avoided and the true data distribution even recovered, provided the mixture weight of real data remains non-zero over time and is paired with sufficient sample sizes. This mechanism consistently enhances performance across diverse model classes, offering both theoretical guarantees and practical guidance for sustainable iterative training.

Large language models often produce semantically homogeneous outputs in open-ended generation tasks, failing to meet diversity requirements. This work proposes a unified framework that, for the first time, systematically characterizes the design space of test-time diversity methods by automatically injecting controllable diversity into an intermediate latent representation and conditioning final response generation on this diverse representation. The approach integrates representation-level guidance, conditional language modeling, and a transfer score—quantifying the influence of source diversity on model outputs—for joint optimization. Experimental results across five open-ended tasks and four backbone architectures demonstrate that the proposed framework substantially enhances output diversity while maintaining generation quality on par with baseline models.

研究通过信息论方法分析大语言模型生成输出的条件熵，发现其低于训练数据，并提出一种后处理机制增加输出多样性。