FitVTON: Fit-aware Virtual Try-On via Body-Garment Size Control

2026-06-10

AI Summary

Existing virtual try-on methods predominantly rely on 2D image inpainting, which struggles to model the realistic draping relationship between garments and human body shapes, often yielding physically implausible results. This work introduces, for the first time, a body-shape-aware sizing control mechanism into diffusion models by encoding garment–body dimensional relationships through structured text prompts and training on simulated try-on triplets generated from a parametric clothing model. A dual auxiliary head is designed to predict masks for both clothed and exposed body regions, complemented by texture refinement to enhance realism. Furthermore, the authors propose a visual-language-model-based protocol for fit assessment and introduce FittingEffect3K, a real-world dataset for evaluating garment fit. Experiments demonstrate that the proposed method significantly improves garment adherence, size accuracy, and body-shape fidelity while maintaining high image quality, outperforming state-of-the-art approaches on both subjective and objective metrics.

Abstract

While diffusion-based virtual try-on has achieved impressive visual realism, most methods treat the task as 2D inpainting, prioritizing texture preservation over physical plausibility. Consequently, they often produce plausible-looking images that fail to reflect authentic garment fit across diverse body shapes. We present FitVTON, a Fit-aware virtual try-on model on different bodies in the wild. FitVTON encodes garment-body size through structured text prompts, and learn from simulated try-on triplets from parameterized garment model. To improve the fitting effects over garment silhouettes, we introduce two auxiliary head to predict the masks for both the garment and the exposed body. We further introduce a texture rectification stage to improve realistic appearance from simulated data. To evaluate the fitting fidelity, we curate a real-world dataset, FittingEffect3K, combining VLM-based scoring protocol. Both subjective and quantitive experiments show that FitVTON demonstrate authentic fitting fidelity, with significant sizing accuracy and shape preservation over state-of-the-art methods while maintaining competitive image quality. Project Page: https://zenoning.github.io/FitVTON/.

Problem

• virtual try-on
• garment fit
• body shape
• physical plausibility
• size control

Innovation

• virtual try-on
• garment-body size control
• diffusion model
• fitting fidelity
• texture rectification