DiffuMural: Restoring Dunhuang Murals with Multi-scale Diffusion

2025-04-13

AI Summary

Addressing the challenges of large-area damage, scarce training samples, and stringent aesthetic consistency requirements in Dunhuang mural restoration, this paper proposes a multi-scale collaborative diffusion inpainting framework. It integrates ControlNet-based conditional guidance, cross-scale feature alignment, and cycle-consistency loss to ensure stylistic unity, geometric fidelity, and cultural-semantic preservation. We introduce the first four-dimensional evaluation metric for mural restoration—comprising factual accuracy, texture detail, semantic coherence, and visual coherence—and uniquely incorporate a humanistic value dimension. Leveraging domain-specific data augmentation and fine-tuned pre-trained diffusion models, our method achieves state-of-the-art performance on 23 authentic Dunhuang murals. Quantitative metrics and expert aesthetic assessments both demonstrate superiority over existing approaches, successfully recovering mineral pigment textures, rhythmic flying-ap Sarira line work, and historical contextual consistency.

Technology Category

• Computer Vision: Diffusion Models for Vision
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Crowd Sourcing and Human Computation

Application Category

• Web Mining and Content Analysis: Large pretrained models with web data
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics

Abstract

Large-scale pre-trained diffusion models have produced excellent results in the field of conditional image generation. However, restoration of ancient murals, as an important downstream task in this field, poses significant challenges to diffusion model-based restoration methods due to its large defective area and scarce training samples. Conditional restoration tasks are more concerned with whether the restored part meets the aesthetic standards of mural restoration in terms of overall style and seam detail, and such metrics for evaluating heuristic image complements are lacking in current research. We therefore propose DiffuMural, a combined Multi-scale convergence and Collaborative Diffusion mechanism with ControlNet and cyclic consistency loss to optimise the matching between the generated images and the conditional control. DiffuMural demonstrates outstanding capabilities in mural restoration, leveraging training data from 23 large-scale Dunhuang murals that exhibit consistent visual aesthetics. The model excels in restoring intricate details, achieving a coherent overall appearance, and addressing the unique challenges posed by incomplete murals lacking factual grounding. Our evaluation framework incorporates four key metrics to quantitatively assess incomplete murals: factual accuracy, textural detail, contextual semantics, and holistic visual coherence. Furthermore, we integrate humanistic value assessments to ensure the restored murals retain their cultural and artistic significance. Extensive experiments validate that our method outperforms state-of-the-art (SOTA) approaches in both qualitative and quantitative metrics.

Problem

• Restoring large defective areas in ancient murals with limited training samples
• Ensuring restored murals meet aesthetic standards for style and seam details
• Developing evaluation metrics for heuristic image complements in mural restoration

Innovation

• Multi-scale convergence with ControlNet
• Collaborative Diffusion mechanism
• Cyclic consistency loss optimization