Distilling Reasoning Traces into Advisory Prompts for Software Engineering Tasks

2026-08-01

AI Summary

This work addresses the vulnerability of large language models to defective code in their training data, which often leads to erroneous code generation. While chain-of-thought reasoning can mitigate such errors, it incurs substantial computational overhead. The authors propose a lightweight error mitigation strategy that distills general and concise advisory prompts from large models by analyzing the discrepancy in error patterns between small models with and without reasoning enabled. These prompts are then injected into the input to enhance performance. Inspired by human cognitive mechanisms of learning from mistakes and internalizing rules, this approach requires no additional training and enables cross-model transferability. Experiments demonstrate that the method significantly reduces code error rates across multiple medium- and small-scale models, with certain prompts exhibiting strong generalization and substantially decreasing reliance on costly reasoning modes.

Abstract

Language models are widely used for generating and otherwise processing code (e.g., identifying code hallucinations, possible inputs, or predicting outputs); however, LLMs can make mistakes, which can be serious. One key issue is that models are trained on (still) largely human-written, and thus imperfect, code; it's not easy to find sufficiently large code corpora that are entirely free of bugs. Thus, other inference-time ways of reducing LLM errors, without additional training, are desirable. "Reasoning" or "thinking" modes, exposed as a togglable feature by hybrid reasoning models, do reduce errors; however, reasoning consumes additional resources. This paper asks if better performance can be achieved without always incurring the cost of reasoning. Human students of programming learn to avoid mistakes by (a) identifying them, (b) reflecting upon the cognitive lapses that led to them (essentially, "thinking through" the errors), (c) inferring general rules or lessons from these reflections, and (d) internalizing these lessons into rules. In tutorial sessions with an instructor, this is a common Socratic interaction. Examples of such internalizable rules might include the nugget "Before coding, restate the requirements to clarify them." Inspired by this process, this paper describes an approach where we first identify examples in which "thinking mode" in a (low-resource) LLM avoids errors. These errors, and their avoidance via "thinking" in the same LLM, are then examined by a bigger LLM to generate summary explanations; these are then summarized by a large LLM into brief advisory prompts. This approach works on many modest-sized models; in some cases, the "advisory prompts" thus learned can also be gainfully transferred to other models. We also present investigations into the nature of coding errors that language models make, and a characterization of when this approach can be helpful.

Problem

• code generation
• language model errors
• reasoning mode
• software engineering tasks
• error reduction

Innovation

• reasoning distillation
• advisory prompts
• error mitigation
• code generation
• transferable guidance