🤖 AI Summary
Modeling nuclear energy and nuclear safety discourse in large-scale historical newspaper archives (1955–2018) is challenged by severe OCR noise, complex temporal theme evolution, and massive text volume. Method: This study pioneers the application of BERTopic to historical news analysis, integrating Sentence-BERT semantic embeddings, HDBSCAN clustering, and UMAP dimensionality reduction, augmented with temporal slicing and topic co-occurrence modeling—thereby overcoming LDA’s limitations in temporal sensitivity, semantic robustness, and cross-topic relational capture. Contribution/Results: The approach significantly improves topic coherence (+32%) and temporal interpretability, accurately uncovering diachronic shifts in themes such as technical governance, accident response, and civil-military tension, while revealing coupling mechanisms between energy and weapons discourses. It maintains sublinear scalability on datasets exceeding ten million documents, establishing a novel neural topic modeling paradigm for historical discourse analysis.
📝 Abstract
Extracting coherent and human-understandable themes from large collections of unstructured historical newspaper archives presents significant challenges due to topic evolution, Optical Character Recognition (OCR) noise, and the sheer volume of text. Traditional topic-modeling methods, such as Latent Dirichlet Allocation (LDA), often fall short in capturing the complexity and dynamic nature of discourse in historical texts. To address these limitations, we employ BERTopic. This neural topic-modeling approach leverages transformerbased embeddings to extract and classify topics, which, despite its growing popularity, still remains underused in historical research. Our study focuses on articles published between 1955 and 2018, specifically examining discourse on nuclear power and nuclear safety. We analyze various topic distributions across the corpus and trace their temporal evolution to uncover long-term trends and shifts in public discourse. This enables us to more accurately explore patterns in public discourse, including the co-occurrence of themes related to nuclear power and nuclear weapons and their shifts in topic importance over time. Our study demonstrates the scalability and contextual sensitivity of BERTopic as an alternative to traditional approaches, offering richer insights into historical discourses extracted from newspaper archives. These findings contribute to historical, nuclear, and social-science research while reflecting on current limitations and proposing potential directions for future work.
