document layout analysis

Medieval manuscript images suffer from a scarcity of fine-grained, semantically rich annotations, hindering the application of instance segmentation and multimodal models in cultural heritage. Method: We propose a zero-shot, model-driven framework for page-level, instance-level annotation of medieval manuscripts. It integrates state-of-the-art instance segmentation, zero-shot vision-language models, and multimodal deep learning to achieve fully automatic region segmentation, content recognition, and semantic description of manuscript pages. Crucially, it systematically uncovers “invisible” graphical and textual elements—such as steganographic text, micro-ornaments, and stylized borders—that elude conventional manual annotation. Contribution/Results: The resulting dataset features pixel-accurate instance masks with cross-modal semantic alignment (e.g., image–text pairs). It significantly improves downstream models’ accuracy and generalization on rare visual concepts in medieval imagery, establishing a scalable, low-dependency annotation paradigm for medieval image computing.

This work addresses the challenge of accurately segmenting handwritten and printed text in document digitization under the computational constraints of edge devices, where existing deep learning approaches incur high computational costs and are difficult to deploy in lightweight settings. To overcome this, the authors propose a novel lightweight segmentation framework that operates without deep neural networks. The method first extracts semantically coherent text regions via sentence-level connected component segmentation, then introduces a region-aware handwriting descriptor (RHD) to effectively capture the variability inherent in handwriting. Finally, a conventional classifier is employed for efficient discrimination between handwritten and printed text. Evaluated on both the newly curated MAD-HPTS dataset and the public PHD-AS benchmark, the approach outperforms state-of-the-art methods, achieving over 8× inference speedup with only a 1.4% drop in accuracy, thereby substantially reducing computational overhead and enabling practical edge deployment.

This work addresses logical page segmentation in historical Czech documents, proposing a purely image-based approach to eliminate OCR dependency and mitigate geometric distortion effects on evaluation. We introduce the first large-scale historical Czech logical layout dataset—spanning newspapers, dictionaries, and manuscripts from the 18th–20th centuries—comprising 8,449 pages and 78,863 paragraph-level annotations. We formally define the image-level logical segmentation task and propose a foreground-text-pixel-only evaluation paradigm. Methodologically, we decouple logical segmentation from OCR and geometric priors: integrating U-Net or Mask R-CNN for text detection, graph neural networks to model inter-paragraph semantic relationships, and handwriting-robust preprocessing. Our method achieves a paragraph-level F1 score of 0.82 on TextBite, outperforming OCR-dependent baselines by 17%. We publicly release the dataset, code, and evaluation framework, establishing the first dedicated benchmark for Central European historical document analysis.

This study addresses the challenges of segmenting and semantically annotating images from 16th–17th century maritime-era shipbuilding manuscripts, which are hindered by extreme data scarcity and the highly specialized nature of the domain. To overcome these obstacles, the work presents a novel integration of SAM2, Florence2, and ChatGPT, augmented with the shipbuilding domain ontology ontoShip and terminology glossary glosShip. This synergistic approach enables accurate automatic image segmentation and semantic annotation tailored to historical technical illustrations. The proposed method significantly improves the precision of processing ancient technical documents, thereby establishing an effective pathway for intelligent retrieval and knowledge extraction from historical maritime literature.

Manual annotation of medieval charters is inefficient, verification is time-consuming, and pixel-level segmentation struggles to balance accuracy with interpretability. Method: This paper proposes a lightweight object detection paradigm tailored for diplomatic analysis—modeling document structure segmentation as a bounding-box detection task. It introduces a novel calibration-card-based pixel-to-physical-scale mapping mechanism enabling sub-millimeter length regression (error <5%), and designs a domain-specific ontology-inspired class hierarchy to enhance crowdsourcing annotation consistency and expert annotation reuse. Contribution/Results: Experiments demonstrate over 60% improvement in annotation efficiency, segmentation quality on par with or surpassing fully supervised semantic segmentation, and significantly reduced time investment by paleographers in data correction. The approach establishes an interpretable, quantifiable, and scalable framework for intelligent historical document processing.

This study addresses the challenge of automatically classifying large-scale scanned historical document pages containing diverse visual elements such as text, tables, and figures. The authors construct a dataset of over 48,000 annotated images and collaborate with domain experts to define an 11-category visual content taxonomy. They systematically evaluate multiple models, including a handcrafted-feature-based random forest baseline, fine-tuned EfficientNetV2, RegNetY, Vision Transformer (ViT), DiT, and multimodal CLIP. Among these, RegNetY-16GF achieves a Top-1 accuracy of 99.16% and demonstrates over 90% prediction agreement with other models on more than 640,000 unlabeled pages, substantially outperforming CLIP. This work presents the first systematic comparison of CNNs, Transformers, and CLIP in terms of deployment consistency on large-scale historical documents, providing high-accuracy classification to support downstream OCR and structural analysis.

To address the low accuracy and poor robustness of table row/column boundary detection under low-resolution and noisy conditions, this paper proposes a mask-based multi-scale 1D signal modeling approach. It transforms row/column boundaries into filterable 1D signals, employs progressive-variance Gaussian convolution to suppress noise, and introduces an adaptive statistical thresholding mechanism for precise structural coordinate localization. We pioneer the “mask → 1D signal” modeling paradigm, supporting zero-padding and scaling to accommodate multi-resolution inputs. Integrated into TableNet, our method achieves a Cell-Aware Segmentation Accuracy (CASA) of 76% on PubLayNet-1M—up from 67%—demonstrating substantial improvement in structural parsing performance under image degradation scenarios.

This study addresses the challenge of high-precision, non-invasive reconstruction of fragile paper fragments in cultural heritage by proposing a human–robot collaborative real-time reconstruction system. The system integrates a vacuum-based collaborative robot with the detector-free feature matching algorithm SE2-LoFTR, enabling vision-guided fragment alignment and assembly in either manual or fully automatic modes. It innovatively combines AI-driven analysis—leveraging image segmentation and local feature matching—with a safe vacuum gripper mechanism and high-accuracy robotic control. Experimental results demonstrate a repeatability positioning accuracy of 0.57 mm on fragments as small as 8 cm² and confirm the superior robustness of SE2-LoFTR under conditions involving rotation, scaling, and partial damage.

This work addresses the challenge of accurately and efficiently digitizing complex documents containing handwritten content, irregular tables, and heterogeneous layouts—tasks that remain difficult for conventional OCR systems and current large language models. The authors propose an interactive document digitization system that integrates layout-aware parsing, OCR, and a large language model, enhanced by a user-in-the-loop correction propagation mechanism. Leveraging layout-aware inference, the system automatically generalizes user edits or natural language instructions applied to a local region to structurally similar regions across the document. In a user study (n=12), this approach significantly improved correction efficiency, reduced repetitive manual operations, and enabled more controllable and effective reconstruction of document structure and content.

This study addresses the challenging task of character detection and recognition in handwritten forms, which remains difficult due to structural complexity and heavy reliance on extensive manual annotations. The authors propose an end-to-end deep neural network approach that unifies character detection and classification into a single task, thereby eliminating the dependency on manually annotated data inherent in conventional two-stage pipelines. By synthesizing training data using the EMNIST dataset combined with realistic form layouts, the model achieves strong generalization without requiring real-world labeled examples. Evaluated on actual handwritten examination forms, the method attains an overall character recognition accuracy of 88.28%, substantially outperforming existing two-stage approaches.