Named Entity Recognition in Historical Italian: The Case of Giacomo Leopardi's Zibaldone

2025-05-26

AI Summary

Named entity recognition (NER) for 19th-century Italian historical texts—such as Leopardi’s *Zibaldone*—faces unique challenges including orthographic variation, structural fragmentation, and OCR-induced noise. Method: We introduce the first publicly available, fine-grained NER benchmark for historical Italian humanities texts, comprising 2,899 manually annotated entities across three categories: persons, locations, and literary works. We evaluate domain-adapted BERT-base-it (fine-tuned on historical data) against instruction-tuned LLaMA-3.1, and establish a reproducible experimental framework with rigorous human validation. Results: BERT-base-it achieves 78.4% F1, substantially outperforming LLaMA-3.1 (<52% F1), especially on literary references. Our work establishes the first robust baseline for historical Italian NER and demonstrates that compact, domain-specialized models exhibit superior robustness and practical utility over general-purpose large language models in this low-resource, noisy domain.

Technology Category

• Natural Language Processing: Question Answering
• Machine Learning: Large Multimodal Models (LMMs)
• Application Domains: Humanities & Computational Social Science

Application Category

• Search and Retrieval-Augmented AI: Multilingual and cross-lingual Web search
• Web Mining and Content Analysis: Large pretrained models with web data
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

The increased digitization of world's textual heritage poses significant challenges for both computer science and literary studies. Overall, there is an urgent need of computational techniques able to adapt to the challenges of historical texts, such as orthographic and spelling variations, fragmentary structure and digitization errors. The rise of large language models (LLMs) has revolutionized natural language processing, suggesting promising applications for Named Entity Recognition (NER) on historical documents. In spite of this, no thorough evaluation has been proposed for Italian texts. This research tries to fill the gap by proposing a new challenging dataset for entity extraction based on a corpus of 19th century scholarly notes, i.e. Giacomo Leopardi's Zibaldone (1898), containing 2,899 references to people, locations and literary works. This dataset was used to carry out reproducible experiments with both domain-specific BERT-based models and state-of-the-art LLMs such as LLaMa3.1. Results show that instruction-tuned models encounter multiple difficulties handling historical humanistic texts, while fine-tuned NER models offer more robust performance even with challenging entity types such as bibliographic references.

Problem

• Adapting NER techniques for historical Italian texts
• Evaluating LLMs on 19th-century scholarly notes
• Addressing orthographic variations in Zibaldone dataset

Innovation

• Uses BERT-based models for historical NER
• Evaluates LLMs on Italian historical texts
• Creates dataset from 19th century scholarly notes