On the Impact of Requirement Smells in LLM-Based Code Generation

2026-09-24

AI Summary

This study addresses the gap in empirical research regarding how requirements smells affect the functional correctness of code generated by large language models (LLMs). Based on an established taxonomy, we systematically inject semantic, syntactic, and lexical smells into four benchmarks to quantitatively analyze their impact on test pass rates. Furthermore, this work presents the first systematic comparison of the differential effects of such smells between automated traceability and code generation tasks. Our findings reveal that code generation is substantially more sensitive to requirements quality, with increasing smell density significantly degrading functional correctness. We demonstrate that high-quality requirements constitute a necessary yet insufficient condition for successful generation, providing critical empirical evidence for LLM-assisted software engineering.

Abstract

Software requirements are typically incorporated into prompts used in LLM-assisted software development. Recent work has shown that requirement smells can affect automated traceability between requirements and code, but empirical evidence on their effects in code generation remains limited. To address this gap, we build upon a prior study on automated traceability by reusing its dataset and requirement smell taxonomy, while extending it to evaluate the functional correctness of LLM-generated code. Using a benchmark consisting of requirements and corresponding system tests for four applications, we progressively introduced semantic, syntactic, and lexical smells into otherwise clear requirements and analyzed their influence on generated implementations. Our results suggest that increasing \textit{smell density} was generally associated with lower test-suite-based functional correctness, although non-smelly requirements could still produce faulty code. We also found that different smell categories had similar effects. These findings provide additional empirical evidence of the importance of requirement quality in LLM-assisted code generation, while showing that high-quality requirements alone do not guarantee correctness, as these depends on several factors, including the LLM. Compared with previous work, our results suggest that the impact of requirement smells depends on the software engineering task: whereas their effects on traceability were modest, code generation appears more sensitive. Overall, this work motivates further investigation into task-dependent quality effects in LLM-assisted software engineering.

Problem

• Requirement Smells
• LLM-based Code Generation
• Functional Correctness
• Software Requirements
• Empirical Study

Innovation

• Requirement Smells
• LLM-based Code Generation
• Functional Correctness
• Smell Density
• Task-Dependent Quality