Parallelly Tempered Generative Adversarial Networks

2024-11-18

arXiv.org

AI Summary

GAN training often suffers from mode collapse and high gradient variance due to multimodal target distributions, compromising generation quality and stability. To address this, we propose a parallel GAN training framework that constructs a sequence of tempered distributions via convex interpolation—introducing statistical parallel tempering into GANs for the first time. This enables cooperative optimization of the generator across multiple tempered distributions. We theoretically establish that tempering substantially reduces gradient estimation variance and extend the framework to fair synthetic data generation. Our method integrates tempered distribution construction, multi-objective optimization, and fairness-aware constraint modeling. Empirically, it outperforms state-of-the-art GANs on both image and tabular data synthesis tasks, significantly mitigating mode collapse while improving sample diversity and fidelity. Moreover, it supports controllable fair synthesis with explicit fairness guarantees.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Generation
• Machine Learning: Multimodal Learning

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

A generative adversarial network (GAN) has been a representative backbone model in generative artificial intelligence (AI) because of its powerful performance in capturing intricate data-generating processes. However, the GAN training is well-known for its notorious training instability, usually characterized by the occurrence of mode collapse. Through the lens of gradients' variance, this work particularly analyzes the training instability and inefficiency in the presence of mode collapse by linking it to multimodality in the target distribution. To ease the raised training issues from severe multimodality, we introduce a novel GAN training framework that leverages a series of tempered distributions produced via convex interpolation. With our newly developed GAN objective function, the generator can learn all the tempered distributions simultaneously, conceptually resonating with the parallel tempering in Statistics. Our simulation studies demonstrate the superiority of our approach over existing popular training strategies in both image and tabular data synthesis. We theoretically analyze that such significant improvement can arise from reducing the variance of gradient estimates by using the tempered distributions. Finally, we further develop a variant of the proposed framework aimed at generating fair synthetic data which is one of the growing interests in the field of trustworthy AI.

Problem

• Addresses GAN training instability from mode collapse
• Reduces gradient variance using tempered distributions
• Improves synthetic data generation for fairness

Innovation

• Parallel tempering for GAN training stabilization
• Convex interpolation creates tempered distributions series
• Reduces gradient variance via multi-distribution learning