Advancing Software Quality: A Standards-Focused Review of LLM-Based Assurance Techniques

2025-05-19

AI Summary

This study addresses the critical lack of standard alignment and auditability of large language models (LLMs) in software quality assurance (SQA). We propose the first fine-grained mapping framework that systematically links eight LLM-driven SQA capabilities—such as requirements validation, defect detection, and test generation—to six major international quality standards: ISO/IEC 12207, ISO/IEC 25010, ISO/IEC 5055, ISO 9001, CMMI, and TMM. Our methodology integrates semantic parsing of standard clauses, a software engineering knowledge graph, compliance alignment assessment, and integration with open-source toolchains. Key contributions include: (1) a reusable LLM-SQA–standards mapping matrix; (2) an AI governance paradigm balancing automation efficiency with process maturity; and (3) empirical validation across three industrial case studies, demonstrating feasibility and advancing standardized, compliant deployment of AI-augmented SQA.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Planning, Routing, and Scheduling: Planning with Language Models

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Software Quality Assurance (SQA) is critical for delivering reliable, secure, and efficient software products. The Software Quality Assurance Process aims to provide assurance that work products and processes comply with predefined provisions and plans. Recent advancements in Large Language Models (LLMs) present new opportunities to enhance existing SQA processes by automating tasks like requirement analysis, code review, test generation, and compliance checks. Simultaneously, established standards such as ISO/IEC 12207, ISO/IEC 25010, ISO/IEC 5055, ISO 9001/ISO/IEC 90003, CMMI, and TMM provide structured frameworks for ensuring robust quality practices. This paper surveys the intersection of LLM-based SQA methods and these recognized standards, highlighting how AI-driven solutions can augment traditional approaches while maintaining compliance and process maturity. We first review the foundational software quality standards and the technical fundamentals of LLMs in software engineering. Next, we explore various LLM-based SQA applications, including requirement validation, defect detection, test generation, and documentation maintenance. We then map these applications to key software quality frameworks, illustrating how LLMs can address specific requirements and metrics within each standard. Empirical case studies and open-source initiatives demonstrate the practical viability of these methods. At the same time, discussions on challenges (e.g., data privacy, model bias, explainability) underscore the need for deliberate governance and auditing. Finally, we propose future directions encompassing adaptive learning, privacy-focused deployments, multimodal analysis, and evolving standards for AI-driven software quality.

Problem

• Enhancing SQA with LLMs for automation and compliance
• Mapping LLM-based methods to established quality standards
• Addressing challenges like bias and privacy in AI-driven SQA

Innovation

• LLMs automate SQA tasks like code review
• AI solutions comply with ISO quality standards
• LLMs enhance defect detection and test generation