Reviving Cultural Heritage: A Novel Approach for Comprehensive Historical Document Restoration

2025-07-07

AI Summary

Historical documents suffer severe degradation—such as tearing, water staining, and oxidation—leading to drastically reduced readability and OCR accuracy. Existing methods are limited by single-modality processing and insufficient local restoration capability. This paper introduces FPHDR, the first comprehensive dataset for full-page historical document restoration, and AutoHDR, a modular end-to-end framework. AutoHDR innovatively integrates OCR-guided character-level damage localization, vision-language joint text prediction, and block-wise autoregressive appearance restoration, enabling multi-stage human-in-the-loop optimization. Trained on a hybrid of synthetic and real degraded images, it jointly restores textual semantics and visual morphology with high fidelity. On severely degraded documents, OCR accuracy improves from 46.83% to 84.05%, reaching 94.25% with minimal human intervention—substantially outperforming state-of-the-art methods and providing a practical technical foundation for digital preservation of cultural heritage.

Technology Category

• Computer Vision: Multi-modal Vision
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Historical documents represent an invaluable cultural heritage, yet have undergone significant degradation over time through tears, water erosion, and oxidation. Existing Historical Document Restoration (HDR) methods primarily focus on single modality or limited-size restoration, failing to meet practical needs. To fill this gap, we present a full-page HDR dataset (FPHDR) and a novel automated HDR solution (AutoHDR). Specifically, FPHDR comprises 1,633 real and 6,543 synthetic images with character-level and line-level locations, as well as character annotations in different damage grades. AutoHDR mimics historians' restoration workflows through a three-stage approach: OCR-assisted damage localization, vision-language context text prediction, and patch autoregressive appearance restoration. The modular architecture of AutoHDR enables seamless human-machine collaboration, allowing for flexible intervention and optimization at each restoration stage. Experiments demonstrate AutoHDR's remarkable performance in HDR. When processing severely damaged documents, our method improves OCR accuracy from 46.83% to 84.05%, with further enhancement to 94.25% through human-machine collaboration. We believe this work represents a significant advancement in automated historical document restoration and contributes substantially to cultural heritage preservation. The model and dataset are available at https://github.com/SCUT-DLVCLab/AutoHDR.

Problem

• Restoring degraded historical documents with tears and erosion
• Overcoming limitations of single-modality restoration methods
• Automating multi-stage restoration with human-machine collaboration

Innovation

• OCR-assisted damage localization for precise restoration
• Vision-language context text prediction for accuracy
• Patch autoregressive appearance restoration for completeness