Reviving Ancient Paintings via Poem: A Colorization Framework for Aligning Cultural Semantics

2026-07-20

AI Summary

This work addresses the degradation of the classical Chinese aesthetic principle of “unity between poetry and painting” caused by fading in ancient artworks, a problem exacerbated by existing colorization methods that often suffer from modern semantic bias, oversaturation, and color bleeding. To overcome these limitations, we propose PoemColor, a novel framework that uniquely integrates poetic semantics into the task of historical painting colorization. Our approach introduces a Poetic Painting Projector to translate the implicit cultural context of classical poems into authentic color priors, and a Structure-Aware Semantic Attention mechanism to precisely modulate the direction and intensity of semantic guidance within a diffusion model. Leveraging poem-to-palette pretraining and a hybrid dataset combining synthetically degraded images with expert-restored samples, PoemColor achieves state-of-the-art performance, delivering high-fidelity, controllable colorization that preserves both historical authenticity and poetic expressiveness.

Abstract

The irreversible fading of ancient paintings disrupts the "congruence between poems and paintings", a core aesthetic principle where visual imagery harmonizes with literary inscriptions. Although diffusion models provide strong generative priors, restoring historically faithful colors remains difficult: visual restoration is inherently ambiguous, while direct text guidance often causes modern semantic bias, over-saturation, and cross-boundary color leakage. To address this, we propose PoemColor, a poem-guided ancient painting colorization framework. Our method aligns poetic cultural semantics with painting restoration through two key designs. First, the Poetic Painting Projector (P3) converts implicit poetic context into a classical color-aware condition via poem-to-palette pretraining, reducing the ambiguity of poem-to-color mapping. Second, Structure-Aware Semantic Attention (SASA) regulates how poetic color semantics are injected into the diffusion backbone by jointly controlling their propagation direction and regional injection strength. In addition, we construct a hybrid restoration dataset that integrates synthetic degradation with expert-restored artifacts, providing both scalable supervision and real classical color references. Extensive experiments demonstrate that our framework significantly outperforms state-of-the-art methods, delivering controllable colorization that revives both historical authenticity and poetic semantics.

Problem

• ancient painting colorization
• poem-painting congruence
• cultural semantics
• historical authenticity
• color restoration

Innovation

• poem-guided colorization
• diffusion models
• cultural semantics alignment
• structure-aware attention
• ancient painting restoration