Kongzi: A Historical Large Language Model with Fact Enhancement

2025-04-13

AI Summary

Large language models (LLMs) suffer from factual inaccuracies in multi-step historical reasoning, weak cross-temporal-spatial association, and difficulty integrating fragmented historical sources. To address these challenges, we propose HistoLLM—a domain-specific LLM tailored for historical analysis. Methodologically, we introduce a novel fact-augmented reinforcement learning framework that integrates high-fidelity historical corpus fine-tuning with three synergistic mechanisms: historical knowledge injection, fact-consistency constraints, and multi-source historiographical alignment modeling—jointly optimizing factual accuracy and reasoning depth. Experimental results demonstrate that HistoLLM significantly outperforms baseline models on historical question answering and narrative generation tasks, achieving an 18.7% absolute gain in factual accuracy and setting a new state-of-the-art in reasoning coherence. This work establishes a trustworthy AI inference infrastructure for knowledge-intensive historical research.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Natural Language Processing: (Large) Language ModelsHumans and AI: Human-in-the-loop Machine Learning

Application Category

Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved informationSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsUser Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

The capabilities of the latest large language models (LLMs) have been extended from pure natural language understanding to complex reasoning tasks. However, current reasoning models often exhibit factual inaccuracies in longer reasoning chains, which poses challenges for historical reasoning and limits the potential of LLMs in complex, knowledge-intensive tasks. Historical studies require not only the accurate presentation of factual information but also the ability to establish cross-temporal correlations and derive coherent conclusions from fragmentary and often ambiguous sources. To address these challenges, we propose Kongzi, a large language model specifically designed for historical analysis. Through the integration of curated, high-quality historical data and a novel fact-reinforcement learning strategy, Kongzi demonstrates strong factual alignment and sophisticated reasoning depth. Extensive experiments on tasks such as historical question answering and narrative generation demonstrate that Kongzi outperforms existing models in both factual accuracy and reasoning depth. By effectively addressing the unique challenges inherent in historical texts, Kongzi sets a new standard for the development of accurate and reliable LLMs in professional domains.

Problem

• Addresses factual inaccuracies in long reasoning chains of LLMs
• Enhances historical analysis with accurate cross-temporal correlations
• Improves factual accuracy and reasoning depth in knowledge-intensive tasks

Innovation

• Integrates curated high-quality historical data
• Uses novel fact-reinforcement learning strategy
• Enhances factual alignment and reasoning depth