bug classification

Mapping software failures to a structured taxonomy of vulnerability or bug types and assigning observed failures to appropriate, state-specific repair families to support diagnosis, triage, and remediation planning.

bugclassification

12-Month Skill Trend

Momentum and market value over time
Trending
Score
+20 in 12 mo
96
12 mo agoNow
Career
Value
+$12K in 12 mo
$42K/year
12 mo agoNow

Recommended Survey Paper

Quick overview of the field
View more

Triage in Software Engineering: A Systematic Review of Research and Practice

Nov 05, 2025
YZ
Yongxin Zhao
🏛️ Nankai University | Tsinghua University | Microsoft

Software issue triage faces challenges of low operational efficiency and a persistent academic–industrial gap in complex system maintenance. To address this, we conduct a systematic literature review (SLR) spanning 234 English and Chinese publications from 2004 to 2023. This is the first SLR to jointly analyze academic research and industrial practice, revealing three critical bottlenecks: misaligned objectives between academia and industry, lack of standardized evaluation criteria, and practical deployment barriers. We propose a unified triage evaluation framework structured along four dimensions—data, tasks, metrics, and benchmarks—and systematically catalog open-source datasets and empirical methodologies to enable reproducible performance validation. All reviewed literature and supporting resources are publicly released. Our work establishes a foundational theory, provides actionable evaluation tools, and outlines collaborative pathways to bridge the gap between laboratory research and industrial adoption of triage technologies.

Addressing obstacles limiting practical deployment of triage systemsBridging the gap between academic innovation and industrial applicationPrioritizing and assigning issues in complex software systems efficiently

This study addresses the critical gap in systematic understanding of real-world failures in Model Context Protocol (MCP)-based systems, which hinders their reliability and safety. Through large-scale empirical analysis and a practitioner survey, we propose and validate a taxonomy of MCP-related failures comprising five high-level categories, explicitly distinguishing MCP-specific faults from general ones. Our findings reveal the prevalence of each failure type in practice, identify the most error-prone and critical components within MCP systems, and provide actionable insights for building more robust and secure AI software. This work establishes the first comprehensive foundation for failure analysis in MCP-based architectures, offering concrete guidance to improve system dependability.

foundation modelsModel Context Protocolreliability

Must-Read Papers

Most classic and influential ideas
View more

Streamlining Security Vulnerability Triage with Large Language Models

Jan 31, 2025
MJ
Mohammad Jalili Torkamani
🏛️ University of Nebraska–Lincoln | Microsoft | Oracle Labs

To address the inefficiency and high error rate of manual vulnerability triage, this paper proposes an automated vulnerability analysis method leveraging large language models (LLMs), specifically GPT. We introduce CASEY, a novel multi-level contextual fusion prompting framework that jointly models CWE classification and severity assessment—overcoming semantic understanding and cross-vulnerability generalization limitations inherent in traditional rule-based and machine learning approaches. Our method integrates multi-granularity contextual injection, constructs an enhanced NVD dataset with domain-specific curation, and employs a hybrid quantitative–qualitative evaluation strategy. On the enhanced NVD benchmark, CASEY achieves 68.0% accuracy for CWE identification, 73.6% for severity assessment, and 51.2% for the joint task—demonstrating substantial improvements in triage efficiency and decision consistency.

Information ProtectionSoftware SecurityVulnerability Assessment

This study addresses the critical issue of inconsistent results from open-source software vulnerability scanners, which significantly hinders informed supply chain security decisions. The work proposes a novel conceptual framework that characterizes the information flows and root causes of inconsistency within the open-source vulnerability ecosystem, modeling vulnerability management as a distributed information transformation process encompassing creation, standardization, enrichment, and contextual interpretation. By integrating multiple vulnerability data standards and real-world case studies, the analysis systematically identifies four core challenges—identity modeling, version semantics, temporal evolution, and contextual assessment—that underlie result discrepancies. This framework establishes a theoretical foundation and offers practical guidance for reproducible evaluation, accurate interpretation of scanner outputs, and dynamic vulnerability knowledge management.

inconsistent findingsopen-source vulnerability ecosystemsoftware supply chain security

This study addresses the challenges in assessing the completeness of multi-patch vulnerability fixes and the lack of systematic understanding of their root causes and characteristics. Through manual analysis of 1,646 multi-patch repair records, this work proposes the first three-tier classification framework grounded in root causes, revealing the evolutionary patterns of such repairs. By contrasting key features, it clarifies the distinctions between multi-patch and single-patch fixes and evaluates the effectiveness of mainstream vulnerability detection tools in verifying repair completeness. The findings delineate predominant multi-patch repair patterns and associated challenges, expose limitations of current tools, and provide a novel perspective along with an empirical foundation for future research on repair validation.

multi-patch fixesopen-source softwarepatch analysis

A Defect Taxonomy for Infrastructure as Code: A Replication Study

May 02, 2025
FM
Filipe Magno Alves Paiva
🏛️ Federal University of Campina Grande

Existing Infrastructure-as-Code (IaC) defect taxonomies—originally designed for declarative, configuration-language-based tools like Puppet—are of uncertain applicability to programming-language-based IaC (PL-IaC) tools such as Pulumi, Terraform CDK, and AWS CDK. Method: We conducted a large-scale empirical study across open-source repositories and private enterprise codebases (including VTEX and Nubank), qualitatively coding 3,364 PL-IaC defect reports and augmenting classification with the ACID tool for automation and consistency. Contribution/Results: We empirically validate, for the first time, the broad applicability of the original eight-category taxonomy to PL-IaC. Configuration Data defects emerge as the most prevalent category; Idempotency and Security defects—though less frequent—are consistently observed across tools and organizations. This work extends the scope of IaC defect taxonomies, establishing a transferable theoretical framework and actionable foundation for PL-IaC quality assurance.

Analyze defect patterns in 447 open-source and 94 proprietary IaC projectsAssess generalizability of IaC defect taxonomy across diverse PL-IaC toolsValidate persistence of configuration, idempotency, and security defects in IaC

Taxonomic Trace Links: Rethinking Traceability and its Benefits

Apr 29, 2025
WA
Waleed Abdeen
🏛️ Blekinge Institute of Technology | HOCHTIEF GmbH

Software traceability faces practical bottlenecks including inconsistent link granularity, heterogeneous data structures across artifacts, and ambiguous responsibility assignment. To address these, we propose Taxonomic Trace Links (TTL), a paradigm shift from direct link construction to semantic alignment via lightweight domain taxonomies—reframing traceability as “taxonomy-enabled collaboration” rather than “link maintenance.” Our method comprises domain modeling, taxonomy design, semantic mapping of artifacts to taxonomy concepts, and empirical validation. We preliminarily validate TTL using existing taxonomies, identify six critical implementation challenges, and propose a systematic technology evolution roadmap and evaluation framework. TTL is the first approach to treat domain taxonomies as foundational infrastructure for traceability, significantly reducing manual effort and tool dependency. It offers a scalable, industry-ready paradigm for operationalizing traceability in complex software engineering contexts.

Addressing challenges in practical traceability implementationResolving granularity and artifact structure issues in tracingSimplifying trace links via domain-specific taxonomy (TTL)

Latest Papers

What's happening recently
View more

This study addresses previously unexamined runtime failures in Model Context Protocol (MCP) servers, such as accepted configuration parameters that are not enforced, leading to unintended default behaviors and system unreliability. Through manual analysis of 837 runtime failure reports from 473 active MCP repositories, the authors employ a bottom-up open coding approach to construct the first comprehensive taxonomy of MCP runtime failures, encompassing dimensions such as protocol interactions, tool invocations, and state management. The resulting classification comprises 11 top-level categories and 27 subcategories, totaling 73 leaf-level failure types. Validated by 55 developers—each encountering an average of 20 categories—and supported by empirical observations across all categories, the taxonomy demonstrates broad applicability and strong external validity.

Large Language ModelsModel Context Protocolreliability challenges

This work addresses the lack of automated, structured failure-recovery mechanisms in current software engineering agents, which struggle to translate heterogeneous runtime evidence into actionable repair guidance. The paper proposes PROBE, a novel framework that introduces a failure-anchored, structured recovery paradigm. PROBE employs a three-layer architecture—telemetry, diagnosis, and guidance gate—to decouple yet coordinate diagnosis and recovery, enabling non-intrusive integration. By integrating runtime telemetry, multi-signal diagnosis, and evidence-driven bounded guidance generation, PROBE constructs an end-to-end recovery pipeline. Evaluated on 257 unresolved cases, PROBE achieves a Top-1 diagnostic accuracy of 65.37% and a recovery success rate of 21.79%, significantly outperforming the strongest baseline. Its practical feasibility has been validated through deployment in Microsoft’s IcM system.

failure diagnosispost-failure recoveryruntime telemetry

This study addresses the persistent occurrence of software defects after release, particularly in C/C++ and Java systems, whose underlying causes remain poorly understood. Through a large-scale empirical analysis of over 14,000 open-source projects, the work systematically compares pre-release and post-release defect characteristics using multidimensional metrics—including code complexity, size, change frequency, and development history—and employs statistical modeling to uncover key patterns. It reveals for the first time that post-release defects are significantly concentrated in legacy modules that undergo frequent modifications, with their root causes primarily stemming from dynamic evolutionary pressures rather than static code structure. Furthermore, such defects exhibit longer repair cycles and higher complexity, offering empirical grounding for targeted testing strategies and improved reliability assurance.

defect characterizationpost-release defectsresidual faults

A System-Level Taxonomy of Failure Modes in Large Language Model Applications

Nov 25, 2025
VV
Vaishali Vinay
🏛️ Independent Researcher

This paper addresses the under-recognized reliability challenges of large language models (LLMs) in real-world system deployments. Adopting a systems engineering perspective, it establishes the first fault taxonomy for LLM-based applications. Through systematic analysis and multi-case root-cause investigation, the study identifies 15 classes of latent failures—including multi-step reasoning drift, context boundary degradation, erroneous tool invocation, and latent inconsistency—exposing fundamental limitations of current evaluation benchmarks in stability, reproducibility, and workflow integration. The work introduces high-level design principles centered on observability, cost sensitivity, and version evolution, shifting LLM reliability research from a model-centric to a system-integration paradigm. It delivers the first structured fault classification framework and practical guidance for building reliable, maintainable, and auditable LLM software systems.

Addressing system-level reliability challenges in LLM deploymentAnalyzing gaps in evaluation methods for stability and reproducibilityClassifying fifteen hidden failure modes in real-world LLM applications

Software maintenance remains heavily reliant on manual effort, resulting in high costs, low efficiency, and susceptibility to errors. This work proposes the first systematic research framework for transfer-based software maintenance, drawing inspiration from transfer learning. The framework establishes a comprehensive lifecycle model encompassing task identification, source system selection, cross-system data matching and adaptation, and validation of transferred outcomes. It explicitly delineates the core objectives and key challenges at each stage, integrating techniques from software engineering such as knowledge transfer, cross-project data alignment, and context-aware adaptation. By doing so, the framework introduces a novel paradigm for automating software maintenance and lays a solid theoretical foundation for the future development of supporting tools and methodologies.

automated maintenanceknowledge transfermigration-based maintenance

Hot Scholars

MM

Mohammad Masudur Rahman

PhD Candidate at University of Louisiana at Lafayette
Natural Language ProcessingMachine LearningDeep LearningHuman Computer Interaction
YL

Yang Liu

Nanyang Technological University
AgentSoftware EngineeringCyber SecurityTrustworthy AI
AM

Armin Moin

Assistant Professor of Computer Science at the University of Colorado Colorado Springs (UCCS)
Software EngineeringArtificial IntelligenceQuantum-Classical SE & AI
TY

Tingting Yu

Associate Professor, University of Connecticut
Software EngineeringSoftware Testing
PL

Peng Liang

School of Computer Science, Wuhan University
Software EngineeringSoftware ArchitectureEmpirical Software Engineering