Comparison of Feature Learning Methods for Metadata Extraction from PDF Scholarly Documents

2025-01-09

AI Summary

This study addresses the challenge of extracting key metadata (e.g., title, authors, affiliations) from German social science PDFs, which exhibit high template heterogeneity and widespread metadata absence. We systematically benchmark state-of-the-art NLP (BERT), computer vision (YOLO/CNN-based layout analysis), and multimodal approaches (LayoutLMv3, text-layout-image joint modeling) on metadata extraction, establishing the first unified evaluation framework tailored to real-world, high-diversity German academic PDFs—balancing accuracy and efficiency. Experimental results show that multimodal methods achieve up to a 12.7% F1-score improvement over unimodal baselines, with layout-aware models significantly outperforming pure text-based ones. To foster reproducibility and FAIR metadata infrastructure, we publicly release the first benchmark dataset dedicated to German social science PDFs, along with fully documented, end-to-end training and inference code.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Information Extraction
• Data Mining & Knowledge Management: Mining of Visual, Multimedia & Multimodal Data

Application Category

• Web Mining and Content Analysis: Mining multimedia, multimodal, multilingual, cross-lingual Web data
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

The availability of metadata for scientific documents is pivotal in propelling scientific knowledge forward and for adhering to the FAIR principles (i.e. Findability, Accessibility, Interoperability, and Reusability) of research findings. However, the lack of sufficient metadata in published documents, particularly those from smaller and mid-sized publishers, hinders their accessibility. This issue is widespread in some disciplines, such as the German Social Sciences, where publications often employ diverse templates. To address this challenge, our study evaluates various feature learning and prediction methods, including natural language processing (NLP), computer vision (CV), and multimodal approaches, for extracting metadata from documents with high template variance. We aim to improve the accessibility of scientific documents and facilitate their wider use. To support our comparison of these methods, we provide comprehensive experimental results, analyzing their accuracy and efficiency in extracting metadata. Additionally, we provide valuable insights into the strengths and weaknesses of various feature learning and prediction methods, which can guide future research in this field.

Problem

• PDF extraction
• German social sciences
• research discoverability

Innovation

• PDF Information Extraction
• Text and Image Processing
• German Social Science Literature