UVFaceFusion: Fast Multi-view Topologically Consistent Face Reconstruction in the Wild via UV-space Neural Fusion

2026-07-21

AI Summary

Existing methods struggle to simultaneously achieve high-fidelity facial geometry reconstruction and consistent fixed-topology meshes in unconstrained, in-the-wild scenarios. To address this challenge, this work proposes a feedforward multi-view facial reconstruction framework that introduces, for the first time, a mask-aware UV-space neural fusion mechanism. This approach replaces heuristic topological optimization by directly generating high-quality, fixed-topology face meshes in the standard UV space. The method leverages VGGT and Pixel3DMM to extract multi-view point maps and UV correspondences, and enhances generalization through geometry-to-geometry cross-view fusion. Experiments demonstrate that the proposed method achieves state-of-the-art accuracy on multiple public benchmarks and real-world in-the-wild datasets, completing full reconstruction from 16 input views in under three seconds on a single RTX 4090 GPU.

Abstract

Reconstructing high-fidelity facial geometry with an assigned topology is essential for digital avatar creation and animation, yet existing automated methods often trade off geometric fidelity and in-the-wild generalization. We present UVFaceFusion, a feed-forward framework for multi-view, fixed-topology face reconstruction from daily images. Our key idea is to replace heuristic topological optimization with learnable neural fusion in a canonical UV space. Given multi-view images, we first obtain dense point maps and facial UV correspondences of each view using VGGT and Pixel3DMM, respectively. Then, the view-specific point maps are lifted into the canonical UV domain and fused with a novel mask-aware neural fusion network. The network predicts a complete UV-space point map, from which a fixed-topology mesh is directly sampled. Although trained only on Ava-256, UVFaceFusion generalizes well to multiple public benchmarks and in-the-wild captures, benefiting from its canonical UV-space geometry-to-geometry fusion that reduces dependence on dataset-specific appearance and capture conditions. Experiments on various benchmarks show that UVFaceFusion achieves state-of-the-art reconstruction accuracy while reconstructing a mesh from 16 input views in less than 3 seconds on a single RTX 4090. Code is available at https://github.com/grignarder/UVFaceFusion.

Problem

• face reconstruction
• fixed-topology
• multi-view
• in-the-wild
• geometric fidelity

Innovation

• UV-space neural fusion
• fixed-topology face reconstruction
• multi-view face geometry
• canonical UV mapping
• mask-aware fusion network