Thinking to Recall: How Reasoning Unlocks Parametric Knowledge in LLMs

2026-03-10

AI Summary

Although single-hop factual question answering does not require complex reasoning, large language models (LLMs) can enhance recall of parametric knowledge when generating reasoning chains. This work presents the first hypothesis-driven controlled experiments revealing two underlying mechanisms: a non-semantic computational buffering effect and a generative self-retrieval-guided fact activation effect. Through controlled experiments, reasoning trajectory analysis, hallucination detection, and factual consistency evaluation, the study demonstrates that factual hallucinations in intermediate reasoning steps significantly impair final answer accuracy, whereas properly guided, hallucination-free reasoning paths effectively improve performance. These findings deepen our understanding of LLMs’ internal knowledge retrieval mechanisms and offer novel directions for enhancing the factuality of their outputs.

Technology Category

Machine Learning: Large Multimodal Models (LMMs)Natural Language Processing: (Large) Language ModelsKnowledge Representation and Reasoning: Reasoning with Beliefs

Application Category

Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved informationSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMsUser Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

While reasoning in LLMs plays a natural role in math, code generation, and multi-hop factual questions, its effect on simple, single-hop factual questions remains unclear. Such questions do not require step-by-step logical decomposition, making the utility of reasoning highly counterintuitive. Nevertheless, we find that enabling reasoning substantially expands the capability boundary of the model's parametric knowledge recall, unlocking correct answers that are otherwise effectively unreachable. Why does reasoning aid parametric knowledge recall when there are no complex reasoning steps to be done? To answer this, we design a series of hypothesis-driven controlled experiments, and identify two key driving mechanisms: (1) a computational buffer effect, where the model uses the generated reasoning tokens to perform latent computation independent of their semantic content; and (2) factual priming, where generating topically related facts acts as a semantic bridge that facilitates correct answer retrieval. Importantly, this latter generative self-retrieval mechanism carries inherent risks: we demonstrate that hallucinating intermediate facts during reasoning increases the likelihood of hallucinations in the final answer. Finally, we show that our insights can be harnessed to directly improve model accuracy by prioritizing reasoning trajectories that contain hallucination-free factual statements.

Problem

• reasoning
• parametric knowledge
• factual recall
• large language models
• hallucination

Innovation

• reasoning
• parametric knowledge recall
• computational buffer
• factual priming
• hallucination mitigation