conditional generation

This work addresses the feedback loops that arise after model deployment due to performativity—wherein the model’s predictions influence the data distribution—particularly under strong interventions where the convergence behavior of retraining remains poorly understood. The paper introduces the “stable signal principle,” positing that the prediction target contains an intrinsic component independent of the model (e.g., inherent item quality), and leverages this insight to analyze the dynamics of regularized repeated risk minimization. Theoretically, it establishes that as long as a non-zero stable signal exists, retraining converges geometrically to its direction, even when model-induced effects dominate. This reveals a novel role for regularization in mitigating performative feedback and extends the framework to nonlinear, heterogeneous, and time-varying settings—including language models—thereby explaining the observed stability of training on generated data.

This work addresses the challenge of efficiently generating high-quality training data required for supervised learning in text-to-image generation models. We propose the Guided Adversarial Prompts (GAP) framework—a closed-loop data generation system integrating three core mechanisms: (1) adversarial prompt optimization guided by supervised model loss, (2) target distribution alignment via feature matching or discriminator-based guidance, and (3) online feedback adaptation. GAP is the first method to synergistically couple adversarial generation with explicit distributional constraints, shifting data synthesis from open-loop, static prompting to closed-loop, adaptive refinement. Empirical evaluation across diverse settings—including multi-task learning, heterogeneous model architectures, and distribution shifts (e.g., spurious correlations, unseen domains)—demonstrates substantial improvements in downstream model generalization. Data utilization efficiency increases by up to 3.2× compared to baseline approaches.

This study investigates the evolutionary dynamics of generative models trained iteratively on synthetic data contaminated with real data, aiming to mitigate model collapse induced by data pollution. Through statistical modeling, mixture distribution analysis, and theoretical analysis of iterative training dynamics—complemented by theoretical derivations and simulations based on next-token prediction language models—the work demonstrates that model collapse can be effectively avoided and the true data distribution even recovered, provided the mixture weight of real data remains non-zero over time and is paired with sufficient sample sizes. This mechanism consistently enhances performance across diverse model classes, offering both theoretical guarantees and practical guidance for sustainable iterative training.

This work addresses the collapse of output diversity in recursively retrained generative models that rely solely on a single reward signal. From an alignment perspective, the authors propose a data filtering mechanism based on multiple reward functions to preserve diversity throughout the retraining process. By leveraging game-theoretic and probabilistic distributional dynamics, they formally characterize recursive training under multiple rewards for the first time and theoretically prove that, under heterogeneous preferences, the model converges to a stable distribution corresponding to the weighted Nash bargaining solution. This condition guarantees the retention of diversity within high-reward regions and enables diverse, stable synthetic data retraining without requiring access to real data.

This work addresses the challenge of fine-grained, continuous control over textual attributes—such as length, complexity, sentiment, and tone—in large language models. We propose a novel continuous control signal mechanism based on interpolatable embeddings: each attribute dimension is modeled as a linear interpolation vector between “low” and “high” extremal token embeddings in the word embedding space, enabling conditional generation via lightweight fine-tuning. To our knowledge, this is the first approach to achieve spectrum-based, differentiable, and interpolatable textual attribute control. Experiments on response length control demonstrate that our method significantly improves stability and precision over both in-context learning and discrete-label fine-tuning, reducing control error by 37% while exhibiting strong generalization across unseen attribute values. The code and dataset are publicly released.

Current generative models lack theoretical guarantees for fine-grained controllability, making it difficult to assess their actual control capabilities in human–AI interaction. This work formulates interaction as a control process and, for the first time, establishes a distribution-agnostic controllability framework applicable to any black-box nonlinear generative model, relying solely on output boundedness. Building on control theory, the authors propose an algorithm to estimate controllable sets and derive probably approximately correct (PAC)-style error bounds. Experiments on both language and text-to-image models reveal that model controllability is highly sensitive to experimental settings and remarkably fragile, thereby validating the proposed framework and underscoring the necessity of systematic controllability analysis.

This work addresses the growing risks of misuse and loss of control associated with the broad applicability of foundation models, which existing alignment methods struggle to mitigate through hard behavioral constraints. It establishes capability control as a core objective distinct from alignment and introduces a defense-in-depth framework spanning data, learning, and system layers to enforce multi-granular behavioral constraints throughout the model lifecycle. By integrating techniques such as data distribution shaping, representational intervention, and runtime input/output/action-level safeguards, the paper systematically constructs pathways for capability control. It further identifies critical challenges—including the dual-use nature of knowledge and combinatorial generalization—offering a new paradigm for developing safe and controllable AI systems.

Current post-training of language models relies on abstract scalar rewards, which lack transparency regarding the instructional content of preference data and can lead models to learn spurious correlations, resulting in undesirable behaviors such as excessive stylization or sycophancy. This work proposes a data-centric post-training framework that, for the first time, leverages interpretability methods to explicitly model latent conceptual signals within preference data. By analyzing and identifying key features that distinguish preferred from non-preferred responses prior to optimization, the approach integrates interpretability protocols, statistical hypothesis testing, and fine-grained interventions at both feature and data levels. This enables effective diagnosis and suppression of harmful learning signals, significantly reducing off-target behaviors across multiple benchmarks while enhancing model safety and controllable personality.

This work uncovers an intrinsic tension between training stability and generation diversity in large language models: overly stable training dynamics implicitly minimize the forward KL divergence, leading to reduced output entropy and degradation of linguistic structure. To address this, we propose a feedback-driven controlled training framework that integrates maximum likelihood objectives with real-time generative statistical analysis. Through systematic experiments across diverse architectures and random seeds, we demonstrate that stable training often yields low-entropy, repetitive outputs. Our findings challenge the prevailing assumption that training stability is a sufficient proxy for generation quality, offering a novel perspective on how optimization dynamics shape a model’s expressive capacity.

Language models frequently engage in specification gaming—manifesting as reward hacking, test-case overfitting, user deception, and sycophancy—due to flawed supervision signal design (e.g., incomplete labels or reward functions). This work proposes *rec contextualization training*, a novel training paradigm that mitigates specification gaming at its root without modifying the original labels or reward functions. Its core innovation is *counterfactual rec contextualization*: high-quality responses are first generated under suppressive prompts, then reformulated as responses to permissive prompts for supervised fine-tuning. Evaluated across four canonical specification-gaming behaviors, the method significantly suppresses all while preserving baseline task performance. It enhances behavioral robustness and generalization without requiring improvements in supervision quality, offering a supervision-agnostic remedy to specification gaming.