constrained decoding

Existing constrained decoding approaches treat schemas solely as structural constraints, overlooking the potential influence of their linguistic formulation on large language model behavior. This work reframes structured generation as a multi-channel instruction problem, demonstrating that subtle adjustments to schema key wording can implicitly convey instructions to guide model outputs—without altering prompts or model parameters. We systematically reveal, for the first time, that schema phrasing serves as an effective implicit instruction channel. Furthermore, we find significant differences across model families in their sensitivity to prompt-level versus schema-level instructions, with their interaction exhibiting non-additive effects. Experiments show that Qwen substantially benefits from schema-level instructions in mathematical reasoning tasks, whereas LLaMA relies more heavily on prompt-level guidance, and combining both channels does not necessarily yield cumulative performance gains.

This work addresses the low compliance and lack of systematic evaluation of constrained decoding techniques under realistic, complex constraints—particularly JSON Schema. We introduce JSONSchemaBench, the first large-scale structured generation benchmark comprising 10K real-world JSON schemas, and propose a multidimensional evaluation framework assessing compliance, constraint coverage, and output quality across six state-of-the-art methods. Our analysis reveals, for the first time, that compliance rates drop by over 40% for existing frameworks when handling nested, recursive, and conditional constraints. We further identify XGrammar and Outlines as achieving the best trade-off between inference efficiency and generation quality. The benchmark and evaluation suite are fully open-sourced, filling a critical gap in systematic evaluation for structured generation and advancing constrained decoding toward higher reliability and stronger generalization.

This work addresses the pervasive fact hallucination and low verification efficiency in neural language generation by proposing Truth-Aware Decoding (TAD), a semantic safeguarding mechanism that aligns generated text with external knowledge bases in real time during decoding. Methodologically, TAD establishes a constraint-based semantic framework and formally proves the local likelihood optimality of greedy decoding under complete protection; it introduces “knowledge-aware safe quality entropy” as a computable factual risk metric and designs a multi-agent operational calculus—its core implementation rigorously verified using the Lean theorem prover. Experiments demonstrate that TAD significantly reduces hallucination rates across mainstream instruction-tuned models (average reduction: 42.3%) without appreciably increasing generation latency. This work pioneers the deep integration of probabilistic program semantics, formal verification, and decoding-time semantic guarding, establishing a new paradigm for trustworthy large-language-model generation that balances theoretical rigor with engineering practicality.

This work addresses the semantic drift often induced by hard constraints in structured generation, where conventional constrained decoding—while ensuring syntactic validity—can yield outputs that are locally well-formed yet semantically incorrect. To mitigate this issue, the authors propose a training-free, two-stage inference framework: first generating an unconstrained semantic draft, then performing constraint-aware decoding conditioned on this draft to decouple semantics from structure. By conditioning on the draft, the method alleviates the distributional shift caused by rigid constraints and optionally incorporates a multi-draft selection mechanism to further enhance output quality. Evaluated on benchmarks such as GSM8K, the approach improves structured accuracy by up to 24 percentage points (from 15.2% to 39.0%), demonstrating that ensembles of smaller models using this strategy can outperform larger models relying on traditional constrained decoding baselines.

Existing approaches to natural language generation under strict hard constraints—such as the RADNER rules—exhibit significant limitations in both constraint expressivity and adherence. Method: This paper introduces a “constraint-first” framework that systematically integrates constraint programming (CP) into NLP text generation for the first time. It formalizes generation as a discrete combinatorial optimization problem, jointly encoding linguistic features (e.g., n-grams, syllables, character counts) and declarative constraints. A large language model (LLM) then ranks candidate outputs via perplexity-based scoring to identify the optimal solution. Crucially, the method requires no fine-tuning or prompt engineering. Contribution/Results: The framework achieves fully automatic compliance with extremely stringent syntactic and structural constraints. Empirical evaluation in clinical and vision-science domains demonstrates robust generation of large-scale, constraint-satisfying sentences—even under “unreasonably strong” constraints—establishing a novel paradigm for hard-constraint text generation.

This study addresses the overlooked trade-off between output validity and factual correctness when small language models are subjected to hard structural constraints—such as JSON formatting or tool-call schemas—which significantly degrade answer accuracy. The authors introduce "constraint tax," a metric protocol that quantifies the performance penalty imposed by such constraints under fixed model, task, and input conditions, and propose a novel delayed-enforcement paradigm: first allowing unconstrained reasoning, then applying structural constraints post-hoc. Experiments on Qwen2.5 and SmolLM2 models reveal that while hard constraints achieve 100% format compliance, they reduce answer accuracy from 19.7% to 11.0%, with 88.9% of outputs being syntactically valid yet factually incorrect. In calendar-based tool-calling tasks, executable accuracy plummets from 91.5% to 48.0%.

Diffusion-based large language models (dLLMs) struggle to guarantee syntactic correctness when generating context-free grammars such as code or chemical formulas. This work proposes LAVE, the first constraint decoding framework tailored for dLLMs that balances both reliability and efficiency. Leveraging the inherent parallel token prediction capability of dLLMs, LAVE performs lookahead validation at every position during each non-autoregressive generation step, ensuring that intermediate outputs remain extendable into valid sentences. Experimental results across four prominent dLLMs and three benchmarks demonstrate that LAVE significantly improves syntactic correctness over existing methods while introducing negligible computational overhead.

This work addresses the frequent lack of semantic validity in code generated by large language models (LLMs) for software engineering tasks. To this end, it introduces a projection decoding framework that, for the first time, treats graph-based representations as first-class citizens alongside textual sequences during generation. The framework incrementally constructs partial graph structures in parallel with token prediction, directly embedding domain-specific semantics into the decoding process. This integration enables uncertainty modeling, incremental semantic validation, and provable correctness guarantees, thereby establishing a verifiable foundation for LLM-driven software engineering automation. Experimental results demonstrate that the proposed approach significantly improves the semantic validity of generated artifacts in program synthesis tasks.

This work addresses the inefficiency of large language models in code generation, where lengthy natural language constraint prompts incur high token costs without consistently improving constraint adherence. The authors propose replacing conventional prompts with compact, structured constraint headers and conduct the first systematic, multi-model, multi-task study to disentangle the effects of token reduction from constraint compliance. Through multi-round controlled experiments, rule-based scoring, model self-evaluation, and Cliff’s δ effect size analysis, they demonstrate that reducing constraint-related tokens by 71%—and overall prompt length by 25–30%—does not significantly degrade adherence performance. Crucially, compliance is found to depend primarily on constraint type and task domain rather than prompt format, underscoring the importance of deliberate structured constraint design.

This work addresses the gap between decodability and faithfulness in natural language reasoning, where generated rationales may not accurately reflect a model’s true reasoning process. The authors propose a verifier-coupled reasoning framework that interleaves inline assertions within reasoning trajectories and trains an auxiliary consistency head to predict the outputs of programmatic verifiers from hidden states of rationale segments. This approach leverages consistency training both as a representational shaping mechanism and a diagnostic tool. Empirical validation across multiple domains demonstrates its efficacy: LeanCheck achieves perfect directional disentanglement, KataGo commentaries attain 81% accuracy in win-rate bucket prediction, and code-related tasks exhibit a 98.6% coupling rate. Activation patching further reveals causal influences ranging from 73% to 89%, indicating that consistency loss particularly enhances fine-grained assertion fidelity.