Disentangled Clothed Avatar Generation with Layered Representation

2025-01-08

AI Summary

This work addresses key challenges in virtual human generation—namely, poor disentanglement of body, hairstyle, and clothing components; difficulty handling occlusions; and low rendering efficiency. To this end, we propose LayerAvatar, the first feed-forward diffusion model for end-to-end disentangled dressed virtual human synthesis. Methodologically, we introduce a novel layered UV Gaussian feature plane representation that explicitly encodes semantic and geometric structure per component; incorporate an occlusion-aware constraint loss to robustly handle severe inter-component occlusions (e.g., torso occluding limbs); and adopt a single-stage diffusion architecture enabling real-time, high-resolution (≥1024²) rendering. Experiments demonstrate significant improvements over prior methods in disentanglement fidelity, cross-component transferability, and animation compatibility—including facial expression and hand gesture driving. LayerAvatar is the first framework to unify high-fidelity, controllable, and fully disentangled virtual human generation.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Humans and AI: Game Design — Virtual Humans, NPCs and Autonomous Characters
• Machine Learning: Deep Generative Models & Autoencoders

Application Category

• Economics, Online Markets and Human Computation: Economic ramifications for generative AI infrastructure and applications
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Clothed avatar generation has wide applications in virtual and augmented reality, filmmaking, and more. Previous methods have achieved success in generating diverse digital avatars, however, generating avatars with disentangled components (eg, body, hair, and clothes) has long been a challenge. In this paper, we propose LayerAvatar, the first feed-forward diffusion-based method for generating component-disentangled clothed avatars. To achieve this, we first propose a layered UV feature plane representation, where components are distributed in different layers of the Gaussian-based UV feature plane with corresponding semantic labels. This representation supports high-resolution and real-time rendering, as well as expressive animation including controllable gestures and facial expressions. Based on the well-designed representation, we train a single-stage diffusion model and introduce constrain terms to address the severe occlusion problem of the innermost human body layer. Extensive experiments demonstrate the impressive performances of our method in generating disentangled clothed avatars, and we further explore its applications in component transfer. The project page is available at: https://olivia23333.github.io/LayerAvatar/

Problem

• Virtual Character Creation
• Part Segmentation
• Occlusion Handling

Innovation

• LayerAvatar
• Hierarchical Rendering
• Interchangeable Body Parts