🤖 AI Summary
Standard retrieval-augmented generation (RAG) architectures struggle to align with the methodological demands of interpretive disciplines such as history, particularly in their handling of evidentiary use and interpretive norms. This work proposes a novel RAG framework tailored for historical research that decouples retrieval from generation, incorporates temporal window constraints, and employs dual-path recall combining keyword-based and semantic search. It further introduces “Zwischentexte”—interpretive intermediate texts—as a novel mechanism to embed historiographical epistemological principles directly into system design. A transparent, contestable relevance assessment is achieved through an LLM-as-judge approach. Evaluated on a dataset of 102,189 articles from Der Spiegel, the framework demonstrably mitigates period-specific terminological bias and weakly relevant retrievals, substantially enhancing the historiographical validity and explanatory power of generated outputs.
📝 Abstract
Retrieval-Augmented Generation (RAG) is the prevailing architecture for grounding language model outputs in external evidence, yet its dominant evaluation paradigms and default configurations remain oriented toward factual question-answering. For interpretive disciplines such as historical studies, RAG embeds assumptions that conflict with scholarly practice. We introduce HistoRAG, a framework that translates historiographical principles into concrete architectural interventions. Separated retrieval and generation decouples source discovery from interpretation, temporal windowing enforces balanced source representation across the research period as a methodological requirement of historical inquiry, and LLM-as-judge evaluation makes relevance judgments transparent and contestable. We evaluate these interventions using SPIEGELragged, applied to 102,189 articles from Der Spiegel (1950-1979). Each intervention addresses a measurable deficiency in standard RAG: era-specific vocabulary retrieves zero chunks from the 1950s when using 1970s terminology, evidence of the temporal skew that motivates windowing; vector similarity and LLM-assessed relevance correlate only weakly (Spearman rho = 0.275), motivating post-retrieval evaluation; and keyword-based and semantic retrieval surface largely disjoint source pools, motivating an architecture in which both operate as complementary retrieval layers under a shared LLM evaluation filter. We also introduce the concept of Zwischentexte (intermediate texts that function as interpretive proposals rather than findings) as a framework for responsible integration of LLM-generated text into scholarly practice. The architecture offers a model for how domain-specific epistemological commitments can be translated into RAG design decisions, and may transfer to other interpretive disciplines working with large corpora.