SymStep: Symbolic Step Verification for Logical Reasoning

2026-07-25

AI Summary

This work addresses the failure of chain-of-thought (CoT) reasoning in large language models (LLMs) when tackling constraint-dense logical problems, where error accumulation often leads to invalid conclusions. To mitigate this, the authors propose a novel paradigm that integrates symbolic constraint verification with LLM-based stepwise reasoning. The approach decomposes reasoning into atomic, verifiable assertions and employs a lightweight constraint propagator to dynamically validate consistency, block contradictions, and infer implicit facts in real time. Additionally, it leverages the Most Restricted Variable (MRV) heuristic to guide the LLM’s focus toward the most constrained variables during inference. Evaluated on six benchmarks—including subsets of ZebraLogicBench, AR-LSAT, and LGP-14—the method substantially outperforms existing approaches, achieving accuracies of 97%, 100% (versus 87% for CoT), and 100% (where both CoT and Logic-LM score 0%), effectively preventing erroneous loops and propagation of mistakes.

Abstract

Chain-of-thought (CoT) prompting can fail severely on constraint-dense logical reasoning tasks, where unverified errors accumulate silently across steps. We introduce SymStep: an LLM makes one atomic claim at a time (DEDUCE: Alice, pet, Cat), then a lightweight constraint propagator checks the claim for consistency with prior accepted deductions, rejects contradictions, and cascades implied facts automatically. SymStep+G additionally provides MRV guidance after each accepted step, directing the LLM toward the most constrained unresolved variable. On a 35-puzzle retained subset of ZebraLogicBench, a benchmark of 1,000 Einstein-style logic puzzles, Direct and CoT both achieve 0%, while SymStep+G reaches 97%. On AR-LSAT analytical reasoning problems, SymStep achieves 100% vs. CoT's 87%. On LGP-14, SymStep+G achieves 100% vs. 0% for CoT and Logic-LM, the strongest prior symbolic+LLM baseline we compare against. Ablation studies reveal that MRV guidance is a key mechanism for reducing directionless cycling, while consistency checking provides a safety net against explicit contradictions. Across six benchmarks spanning five task domains, SymStep variants match or exceed every baseline on constraint-dense and arithmetic tasks. Experiments on AQUA-RAT algebra confirm the advantage is constraint-density-specific.

Problem

• logical reasoning
• constraint-dense tasks
• error accumulation
• chain-of-thought prompting
• symbolic verification

Innovation

• Symbolic Step Verification
• Constraint Propagation
• Chain-of-Thought Reasoning
• MRV Guidance
• Logical Reasoning