Agent4RE: A Self-Refining Multi-agent Framework for End-to-End Software Requirements Engineering and Benchmarking

2026-10-07

AI Summary

This study addresses the limitations of existing LLM-based requirements engineering approaches, which lack end-to-end process support, benchmark datasets, and multi-agent collaboration mechanisms. To bridge these gaps, this work proposes a self-optimizing multi-agent framework that orchestrates specialized agents and incorporates autonomous iterative refinement with structured human feedback, thereby automating the entire pipeline from requirements elicitation to generation. Furthermore, it constructs RE-E2E, the first real-world end-to-end requirements engineering benchmark dataset. By integrating LLM-as-a-judge evaluation techniques, extensive experiments across eight large language models demonstrate that the proposed framework yields an average 8% improvement in textual metrics and surpasses baseline methods by approximately 0.8 points in evaluation scores, substantiating its practical value for industrial applications.

Abstract

Existing LLM-based approaches for software Requirements Engineering (RE) typically rely on basic prompting strategies or rudimentary agent collaboration, under-utilizing the full potential of multi-agent systems. Meanwhile, available datasets focus on isolated subtasks, such as requirements extraction, classification, and completeness detection, leaving the absence of an end-to-end RE benchmark that spans from requirements elicitation to generation. We present Agent4RE - a self-refining multi-agent RE system that orchestrates specialized agents and incorporates two iterative improvement loops. To support evaluation, we construct RE-E2E - a real-world dataset built from human-written requirement specifications, enabling end-to-end assessment of RE workflows. Building on this foundation, we further propose two enhanced Agent4RE versions that incorporate either autonomous self-refinement or structured human feedback, and analyze their strengths and limitations across different scenarios. Evaluation on 8 Large Language Models (LLMs) demonstrates that all three Agent4RE variants consistently outperform a domain-context-augmented prompting baseline by average 8% in text-based metrics. The two enhanced variants achieve the highest LLM-as-a-judge and human ratings, surpassing two RE baselines by approximately 0.8 points on a four-point scale. This consistent performance establishes Agent4RE as a practical end-to-end RE solution for industrial environments.

Problem

• Requirements Engineering
• Multi-agent Systems
• End-to-End Benchmarking
• Large Language Models

Innovation

• Multi-agent Framework
• Self-Refinement
• Requirements Engineering
• End-to-End Benchmarking
• Large Language Models