ReqGenX: An Empirical Study of Atomic Decomposition, Artifact Regeneration, and Reconstruction for Legacy SRS Documents

2026-07-17

AI Summary

This work addresses the lack of fine-grained evaluation benchmarks for large language models (LLMs) in generating software requirements specifications (SRS), particularly the absence of traceability among source requirements, intermediate artifacts, and generated outputs. The paper introduces the first framework to synthetically reconstruct pre-SRS artifacts with complete traceability chains from legacy SRS documents. By atomizing original SRS statements and leveraging multi-LLM voting for classification, constraint-guided prompting, and judge-guided iterative refinement, the approach produces high-fidelity intermediate artifacts. Experimental results demonstrate strong performance in both faithfulness (AlignScore 0.80–0.99) and downstream reconstructability (SBERT similarity 0.69–0.75), enabling fine-grained evaluation of SRS generation and revealing critical trade-offs among faithfulness, information retention, and artifact completeness.

Abstract

Background: Evaluating automated Software Requirements Specification (SRS) generation is challenging because few datasets provide fine-grained traceability between source requirements, intermediate elicitation artifacts, and generated specifications. Aims: We aim to study whether legacy SRS documents can be transformed into traceable synthetic pre-SRS artifacts that support fine-grained evaluation of LLM-based SRS generation. Method: We conduct an empirical study using ReqGenX, a controlled pipeline that decomposes SRS sections into source-grounded atomic statements, routes atoms to standards-inspired artifact types through multi-LLM plurality voting, and generates artifacts using constrained prompts with iterative judge-guided refinement. We evaluate ReqGenX on seven PURE SRS documents using grounding, quality, information retention, and downstream reconstruction analyses. Results: ReqGenX produces faithful and usable atoms, with median AlignScore values typically between 0.96 and 0.99 and Prometheus scores ranging from 4.34 to 4.85. Generated artifacts remain strongly grounded in their source atoms, with AlignScore values typically between 0.80--0.94 and judge pass rates near 100%; stricter Prometheus evaluation yields pass rates from 54.8% to 97.1%. In a downstream SRS reconstruction case study, artifact-backed atoms remain recoverable from generated SRSs, with SBERT means between 0.69 and 0.75 and AlignScore medians between 0.76 and 0.84. Conclusions: Traceable synthetic pre-SRS artifacts can support more fine-grained evaluation of LLM-based SRS generation, while exposing tradeoffs among faithfulness, information retention, and artifact completeness.

Problem

• Software Requirements Specification
• LLM-based generation
• fine-grained evaluation
• traceability
• legacy SRS documents

Innovation

• atomic decomposition
• artifact regeneration
• traceable SRS evaluation
• LLM-based requirements engineering
• judge-guided refinement