RAGMesh with FaME-G2E: Long-Form Text-Driven 3D Face Generation and Editing

2026-08-10

AI Summary

Existing methods struggle to accurately generate and edit fine-grained geometric details of 3D faces—such as eyebrow tension or cheek contraction—from long textual descriptions. To address this challenge, this work introduces FaME-G2E, a large-scale multimodal dataset, and proposes RAGMesh, a retrieval-augmented framework that integrates text-guided global and regional geometric priors in blendshape space. The framework innovatively combines a multi-scale retrieval fusion (MSRF) module with an adaptive RAG-guided supervision (AdaRAGS) mechanism to achieve precise semantic alignment and localized deformation control. Experimental results demonstrate that the proposed method significantly outperforms current state-of-the-art approaches in terms of local geometric accuracy, text controllability, regional editing precision, and inference efficiency.

Abstract

Text-driven 3D face generation and editing remains challenging due to the difficulty of translating long-form descriptions into fine-grained facial geometry. Existing methods primarily align global textual semantics with facial structures but often struggle to capture subtle local deformations, such as eyebrow tension, cheek contraction, and asymmetric mouth motions, resulting in limited geometric fidelity and editing precision. To facilitate fine-grained text-driven facial modeling, we first construct FaME-G2E, a large-scale multimodal dataset containing detailed text--mesh annotations and paired text--blendshape samples for unified 3D facial generation and editing. Based on this dataset, we propose RAGMesh, a retrieval-augmented framework that leverages text-correlated geometric priors to improve high-fidelity facial synthesis and editing. Specifically, the Multi-Scale Retrieval Fusion (MSRF) module retrieves semantically consistent global and regional facial priors and fuses them in the blendshape space, suppressing conflicting local deformations while preserving coherent deformation patterns. Furthermore, we introduce Adaptive RAG-guided Supervision (AdaRAGS), a region-aware constraint that explicitly aligns textual semantics with corresponding facial regions, enhancing regional controllability and editing accuracy. Extensive experiments on FaME-G2E demonstrate that RAGMesh achieves superior performance over state-of-the-art methods in local geometric accuracy, text-guided controllability, regional editing precision, and inference efficiency. Video demo is available at https://youtu.be/Yr0_XkpWcNk, and the source code and dataset will be released upon paper acceptance.

Problem

• text-driven 3D face generation
• fine-grained facial geometry
• long-form text description
• local deformation
• 3D face editing

Innovation

• RAGMesh
• FaME-G2E
• text-driven 3D face generation
• blendshape-based editing
• retrieval-augmented generation