COutfitGAN: Learning to Synthesize Compatible Outfits Supervised by Silhouette Masks and Fashion Styles

2025-02-12

IEEE transactions on multimedia

AI Summary

This paper introduces a novel task—fashion outfit generation from an arbitrary number of given clothing items—aiming to synthesize visually realistic and stylistically harmonious complementary garments. Methodologically, it proposes the first generative framework for completing outfits from partial inputs, featuring a pyramid-style extractor to model multi-granularity fashion features, and a dual-discriminator joint optimization scheme: a U-Net-based discriminator assesses image realism, while a relational discriminator models cross-item compatibility. Additionally, contour mask supervision is incorporated to enhance fine-grained structural consistency. Evaluated on a large-scale dataset comprising 200K outfits and 800K individual items, the method achieves significant improvements over state-of-the-art approaches across quantitative metrics—including image fidelity, outfit compatibility, and visual similarity—demonstrating both effectiveness and generalizability.

Technology Category

• Computer Vision: Generative Adversarial Networks (GANs) for Vision
• Natural Language Processing: Generation
• Search and Optimization: Combinatorial Optimization

Application Category

• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

How to recommend outfits has gained considerable attention in both academia and industry in recent years. Many studies have been carried out regarding fashion compatibility learning, to determine whether the fashion items in an outfit are compatible or not. These methods mainly focus on evaluating the compatibility of existing outfits and rarely consider applying such knowledge to ‘design’ new fashion items. We propose the new task of generating complementary and compatible fashion items based on an arbitrary number of given fashion items. In particular, given some fashion items that can make up an outfit, the aim of this paper is to synthesize photo-realistic images of other, complementary, fashion items that are compatible with the given ones. To achieve this, we propose an outfit generation framework, referred to as COutfitGAN, which includes a pyramid style extractor, an outfit generator, a UNet-based real/fake discriminator, and a collocation discriminator. To train and evaluate this framework, we collected a large-scale fashion outfit dataset with over 200 K outfits and 800 K fashion items from the Internet. Extensive experiments show that COutfitGAN outperforms other baselines in terms of similarity, authenticity, and compatibility measurements.

Problem

• Generate compatible fashion items
• Design new complementary outfits
• Photo-realistic synthesis using COutfitGAN

Innovation

• Pyramid style extractor used
• UNet-based discriminator implemented
• Large-scale dataset collected