AutoGAN-Distiller: Searching to Compress Generative Adversarial Networks

2020-06-15

International Conference on Machine Learning

AI Summary

Generative Adversarial Network (GAN) generators suffer from excessive model size and lack of general-purpose compression methods for mobile deployment. Method: This paper proposes the first fully automated compression framework tailored specifically for GAN generators. It pioneers the integration of AutoML into GAN compression by jointly optimizing a customized lightweight search space, neural architecture search (NAS), and knowledge distillation—enabling end-to-end discovery of efficient architectures solely from the original generator, without requiring the discriminator or task-specific modifications. The framework is agnostic to GAN architecture and loss function. Results: Evaluated on image-to-image translation and super-resolution tasks, the compressed models achieve several-fold parameter reduction while significantly outperforming existing compression approaches in FID (for generation quality) and PSNR (for reconstruction fidelity), successfully balancing model compactness and perceptual fidelity.

Abstract

The compression of Generative Adversarial Networks (GANs) has lately drawn attention, due to the increasing demand for deploying GANs into mobile devices for numerous applications such as image translation, enhancement and editing. However, compared to the substantial efforts to compressing other deep models, the research on compressing GANs (usually the generators) remains at its infancy stage. Existing GAN compression algorithms are limited to handling specific GAN architectures and losses. Inspired by the recent success of AutoML in deep compression, we introduce AutoML to GAN compression and develop an AutoGAN-Distiller (AGD) framework. Starting with a specifically designed efficient search space, AGD performs an end-to-end discovery for new efficient generators, given the target computational resource constraints. The search is guided by the original GAN model via knowledge distillation, therefore fulfilling the compression. AGD is fully automatic, standalone (i.e., needing no trained discriminators), and generically applicable to various GAN models. We evaluate AGD in two representative GAN tasks: image translation and super resolution. Without bells and whistles, AGD yields remarkably lightweight yet more competitive compressed models, that largely outperform existing alternatives. Our codes and pretrained models are available at this https URL.

Problem

• Generative Adversarial Networks
• Model Compression
• Mobile Devices

Innovation

• AutoGAN-Distiller
• Knowledge Distillation
• AutoML for GAN Compression