🤖 AI Summary
Existing constrained decoding approaches treat schemas solely as structural constraints, overlooking the potential influence of their linguistic formulation on large language model behavior. This work reframes structured generation as a multi-channel instruction problem, demonstrating that subtle adjustments to schema key wording can implicitly convey instructions to guide model outputs—without altering prompts or model parameters. We systematically reveal, for the first time, that schema phrasing serves as an effective implicit instruction channel. Furthermore, we find significant differences across model families in their sensitivity to prompt-level versus schema-level instructions, with their interaction exhibiting non-additive effects. Experiments show that Qwen substantially benefits from schema-level instructions in mathematical reasoning tasks, whereas LLaMA relies more heavily on prompt-level guidance, and combining both channels does not necessarily yield cumulative performance gains.
📝 Abstract
Constrained decoding has been widely adopted for structured generation with large language models (LLMs), ensuring that outputs satisfy predefined formats such as JSON and XML. However, existing approaches largely treat schemas as purely structural constraints and overlook the possibility that their linguistic formulation may affect model behavior. In this work, we study how instruction placement influences model performance in structured generation and show that merely changing the wording of schema keys, without modifying the prompt or model parameters, can significantly alter model performance under constrained decoding. Based on this observation, we propose to reinterpret structured generation as a multi-channel instruction problem, where instructions can be conveyed explicitly through prompts and implicitly through schema keys during decoding. To the best of our knowledge, this is the first work to systematically study how schema key formulation acts as an implicit instruction channel and affects model performance under constrained decoding. Experiments on multiple mathematical reasoning benchmarks show that different model families exhibit distinct sensitivities to these instruction channels: Qwen models consistently benefit from schema-level instructions, while LLaMA models rely more heavily on prompt-level guidance. We further observe non-additive interaction effects between instruction channels, showing that combining multiple channels does not always lead to further improvement. These findings suggest that schema design not only determines output structure, but also carries instruction signals, offering a new perspective on structured generation in LLMs.