chain-of-thought prompting

Complex claim verification suffers from insufficient strategic guidance and diversity in reasoning program generation, alongside difficulties in transitioning from zero-shot to few-shot settings. Method: This paper proposes the first fully automated, human-intervention-free bootstrapping few-shot framework. It explicitly models claim decomposition and information retrieval strategies as structured guidance, integrating program-guided reasoning, strategy-driven program synthesis, bootstrapped demonstration optimization, and execution-based evaluation. Contributions/Results: (1) It enables end-to-end automatic evolution of reasoning programs from zero-shot to few-shot regimes; (2) it significantly improves verification accuracy on complex claims, outperforming existing few-shot baselines; and (3) it enhances interpretability of the reasoning process and cross-domain generalization capability. The framework eliminates manual annotation or prompt engineering, achieving robust performance across diverse claim types and domains through self-improved demonstrations and strategy-aware program generation.

This study investigates how prompt engineering can enhance the performance, reliability, and interpretability of large language models (LLMs) in data analysis tasks while addressing standardization and ethical challenges. We systematically evaluate structured prompting, Chain-of-Thought reasoning, and automated prompt optimization techniques across diverse domains—including healthcare, materials science, finance, and business intelligence—to uncover the interplay among prompt complexity, model architecture, and task performance. Experimental results demonstrate that the proposed approaches yield performance improvements of 6% to over 30% on multiple real-world tasks, underscoring the significant potential of advanced prompting frameworks to strengthen LLMs’ contextual adaptation and practical deployment efficacy.

This work proposes a non-parametric, prompt-level distillation method that balances inference efficiency and interpretability by extracting explicit reasoning paths from large language models and encoding them into structured instructions embedded within the system prompt of a smaller model. Without requiring any fine-tuning, this approach effectively transfers complex reasoning capabilities while introducing negligible latency. Evaluated on StereoSet and Contract-NLI, the method boosts the Macro F1 scores of the Gemma-3 4B model from 57% and 67% to 90.0% and 83%, respectively, achieving state-of-the-art performance. Crucially, the full reasoning logic is preserved in an interpretable form, enabling direct human verification and validation of the model’s decision process.

This work addresses the susceptibility of large language models to hallucination, reasoning drift, and insufficient interpretability in safety-critical tasks by proposing the first structured prompt engineering framework tailored for locally deployed scenarios. The framework explicitly guides models to generate reliable and auditable chains of thought through four complementary dimensions: contextual and scope control, evidence anchoring with traceability, structured reasoning with cognitive control, and safety-specific analytical constraints. Empirical evaluations demonstrate that the approach yields up to a 40% improvement in reasoning performance across multiple model families, maintains consistent gains across model scales, and achieves high inter-annotator agreement in human assessments (Cohen’s κ > 0.80), substantially enhancing reasoning completeness, robustness to interference, and practical utility.

Existing unsupervised chain-of-thought (CoT) prompting methods rely on semantic similarity for in-context example selection, which often introduces noise and suffers from poor interpretability, thereby limiting multi-step reasoning performance. To address this, we propose a reasoning-pattern-based demonstration selection framework. Our approach explicitly models implicit reasoning processes as structured “reasoning patterns”—a novel conceptualization—leveraging large language model priors, prompt engineering, and pattern clustering to construct task-specific, diverse, and interpretable pattern sets that guide reasoning along semantically coherent paths. By decoupling example selection from surface-level semantics and grounding it in latent reasoning structures, our method significantly reduces selection noise. Empirical evaluation across mathematical reasoning, commonsense reasoning, and other multi-step reasoning benchmarks demonstrates consistent performance gains, enhanced robustness, improved transparency, and greater controllability of CoT generation.

Large language models (LLMs) frequently fail in real-world tool invocation due to intent misinterpretation, incorrect parsing of tool documentation, and parameterization errors. To address this, we propose a curriculum-inspired structured reasoning framework that replaces free-form chain-of-thought prompting with guided, template-based reasoning—explicitly decoupling the process into three sequential stages: *intent parsing*, *tool matching*, and *parameter generation*. Our framework employs stepwise structured prompts to jointly model user goals and tool functionalities, thereby enhancing invocation robustness and decision interpretability. Evaluated across multiple state-of-the-art models (e.g., LLaMA-3, Qwen2) and benchmarks (ToolBench, API-Bank), it reduces relative error rates by 3–12% over strong baselines. The core contribution lies in transforming implicit, unstructured reasoning into an explicit, traceable, and modular pipeline—balancing accuracy with transparency and auditability.

This paper identifies a non-monotonic effect of chain-of-thought (CoT) prompting across large language models (LLMs) of varying capability: constrained prompts improve performance on mid-tier models but induce reasoning degradation in stronger models (e.g., GPT-5) due to over-literal interpretation—a phenomenon newly discovered and termed “Prompting Inversion.” Method: We propose “Sculpting,” a rule-based, capacity-aware prompting strategy that dynamically tailors prompt restrictiveness while preserving logical integrity of reasoning steps. Contribution/Results: Evaluated systematically on GSM8K across multiple OpenAI model generations, Sculpting achieves 97.0% accuracy on GPT-4o—surpassing standard CoT—but drops to 94.0% on GPT-5, notably below CoT’s 96.36%, empirically confirming that optimal prompting must simplify as models advance. Our core contribution is uncovering the co-evolutionary principle between prompts and models, establishing both theoretical grounding and a practical framework for adaptive prompt engineering.

This work investigates whether long-chain reasoning capabilities can be effectively elicited in base language models using only a small number of high-quality, human-authored chain-of-thought (CoT) examples or lightweight fine-tuning—without resorting to reinforcement learning or large-model distillation. Method: We propose a synergistic approach integrating prompt engineering, multi-round structured editing, and parameter-efficient fine-tuning, leveraging merely 20 high-precision CoT samples—generated by advanced reasoning models and rigorously validated by human experts—to optimize Qwen2.5-32B. Contribution/Results: Our method yields substantial improvements in mathematical and logical reasoning performance; the fine-tuned model surpasses the larger Qwen2.5-Math-72B-Instruct across multiple benchmarks. Crucially, we empirically demonstrate that carefully curated, human-annotated CoT data exhibits exceptional transfer efficacy for reasoning capability, establishing a cost-effective paradigm for unlocking latent reasoning potential in foundational language models.

This study investigates the internal mechanisms of large language models (LLMs) in scientific reasoning tasks and their dependence on prompting, aiming to enhance model interpretability and safety. To this end, the work proposes prompt optimization as an interpretability tool and introduces a tailored Genetic Evolutionary Pareto Algorithm (GEPA) to systematically optimize scientific reasoning prompts. Combining behavioral analysis with cross-model transfer evaluations, the research reveals that LLMs often rely on “local logic”—model-specific reasoning heuristics that are difficult to generalize across architectures. The findings indicate that performance gains frequently stem from such idiosyncratic structural features, underscoring the importance of characterizing individual model reasoning mechanisms. This insight opens new pathways toward developing safer, more controllable AI systems grounded in a deeper understanding of model-specific behaviors.

Large reasoning models (LRMs) suffer from low inference efficiency and frequent goal divergence; existing training-free methods are constrained by rigid heuristics or intractable analytical requirements. This paper introduces the first training-free reasoning optimization framework, which synergistically integrates evolutionary algorithms with a fine-grained taxonomy of reasoning behaviors to automatically discover and optimize *think-prefixes*—structured prompts that precisely steer reasoning trajectories toward desired goals. The method requires no model fine-tuning, is plug-and-play, and exhibits strong task adaptability and cross-task generalization. Evaluated on DeepSeek-R1-Distill-Qwen-32B, it reduces the StrongREJECT rate from 27.0% to 0.7%, while simultaneously improving inference efficiency, instruction adherence, and safety—effectively alleviating the accuracy–reasoning-length trade-off.