ART-VITON: Measurement-Guided Latent Diffusion for Artifact-Free Virtual Try-On

2025-09-30

AI Summary

Virtual try-on (VITON) faces two key challenges: inaccurate garment alignment within the try-on region and boundary artifacts or identity/background distortion in non-try-on regions. Existing latent diffusion model (LDM)-based approaches struggle to simultaneously ensure geometric consistency and semantic fidelity. This paper proposes ART-VITON, a measurement-guided linear inverse solving framework that formulates virtual try-on as an iterative reconstruction problem under trajectory alignment constraints. It introduces a residual prior initialization to guarantee initial semantic consistency and integrates frequency-domain correction with periodic data-consistency constraints to effectively suppress semantic drift and boundary artifacts. Evaluated on VITON-HD, DressCode, and SHHQ-1.0, ART-VITON achieves significant improvements in visual fidelity and robustness, marking the first method to enable artifact-free, high-fidelity preservation of human identity and background.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Virtual try-on (VITON) aims to generate realistic images of a person wearing a target garment, requiring precise garment alignment in try-on regions and faithful preservation of identity and background in non-try-on regions. While latent diffusion models (LDMs) have advanced alignment and detail synthesis, preserving non-try-on regions remains challenging. A common post-hoc strategy directly replaces these regions with original content, but abrupt transitions often produce boundary artifacts. To overcome this, we reformulate VITON as a linear inverse problem and adopt trajectory-aligned solvers that progressively enforce measurement consistency, reducing abrupt changes in non-try-on regions. However, existing solvers still suffer from semantic drift during generation, leading to artifacts. We propose ART-VITON, a measurement-guided diffusion framework that ensures measurement adherence while maintaining artifact-free synthesis. Our method integrates residual prior-based initialization to mitigate training-inference mismatch and artifact-free measurement-guided sampling that combines data consistency, frequency-level correction, and periodic standard denoising. Experiments on VITON-HD, DressCode, and SHHQ-1.0 demonstrate that ART-VITON effectively preserves identity and background, eliminates boundary artifacts, and consistently improves visual fidelity and robustness over state-of-the-art baselines.

Problem

• Eliminating boundary artifacts in virtual try-on synthesis
• Preserving identity and background in non-try-on regions
• Preventing semantic drift during measurement-guided generation

Innovation

• Measurement-guided latent diffusion framework
• Residual prior-based initialization technique
• Artifact-free sampling with frequency correction