Bridging the Know-Act Gap via Task-Level Autoregressive Reasoning

2026-03-23

AI Summary

This work addresses the “knowing-doing gap” in large language models (LLMs)—their tendency to recognize flaws in questions yet still produce seemingly plausible but incorrect answers. To bridge this gap, we propose DeIllusionLLM, a novel approach that explicitly models the decision between verification and answer generation through a task-level autoregressive framework, unifying discriminative and generative capabilities within a single backbone via a self-distillation mechanism. We further introduce FaultyScience, the first large-scale, cross-disciplinary benchmark designed to systematically expose the prevalence of this gap. Experimental results demonstrate that DeIllusionLLM significantly reduces the rate of erroneous responses under natural prompting while preserving general reasoning performance, thereby validating the effectiveness and scalability of our paradigm in mitigating the knowing-doing gap.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information

Abstract

LLMs often generate seemingly valid answers to flawed or ill-posed inputs. This is not due to missing knowledge: under discriminative prompting, the same models can mostly identify such issues, yet fail to reflect this in standard generative responses. This reveals a fundamental know-act gap between discriminative recognition and generative behavior. Prior work largely characterizes this issue in narrow settings, such as math word problems or question answering, with limited focus on how to integrate these two modes. In this work, we present a comprehensive analysis using FaultyScience, a newly constructed large-scale, cross-disciplinary benchmark of faulty scientific questions. We show that the gap is pervasive and stems from token-level autoregression, which entangles task selection (validate vs. answer) with content generation, preventing discriminative knowledge from being utilized. To address this, we propose DeIllusionLLM, a task-level autoregressive framework that explicitly models this decision. Through self-distillation, the model unifies discriminative judgment and generative reasoning within a single backbone. Empirically, DeIllusionLLM substantially reduces answer-despite-error failures under natural prompting while maintaining general reasoning performance, demonstrating that self-distillation is an effective and scalable solution for bridging the discriminative-generative know-act gap

Problem

• know-act gap
• discriminative-generative gap
• faulty inputs
• autoregressive reasoning
• large language models

Innovation

• know-act gap
• task-level autoregression
• self-distillation
• discriminative-generative alignment
• faulty scientific reasoning