Which Alert Removals are Beneficial?

2026-03-22

AI Summary

This study systematically evaluates the practical impact of static analysis alert removal on code complexity and defect proneness. To address the question of which alert removals are genuinely beneficial, the work proposes three complementary approaches: a randomized controlled trial, natural intervention event mining based on labeling functions, and large-scale intervention analysis combining code commits with supervised learning. The project constructs the first large-scale dataset of alert removals specifically designed for causal inference, thereby advancing the application of causal methods in software engineering. Empirical results demonstrate that effective interventions can reduce complexity in 33% of Python files and lower defect proneness by up to 5.5 percentage points.

Technology Category

Reasoning under Uncertainty: CausalityMachine Learning: Causal LearningKnowledge Representation and Reasoning: Action, Change, and Causality

Application Category

Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphsSecurity and Privacy: Large-scale security measurementsResponsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Context: Static analysis captures software engineering knowledge and alerts on possibly problematic patterns. Previous work showed that they indeed have predictive power for various problems. However, the impact of removing the alerts is unclear. Aim: We would like to evaluate the impact of alert removals on code complexity and the tendency to bugs. Method: We evaluate the impact of removing alerts using three complementary methods. 1. We conducted a randomized controlled trial and built a dataset of 521 manual alert-removing interventions 2. We profiled intervention-like events using labeling functions. We applied these labeling functions to code commits, found intervention-like natural events, and used them to analyze the impact on the tendency to bugs. 3. We built a dataset of 8,245 alert removals, more than 15 times larger than our dataset of manual interventions. We applied supervised learning to the alert removals, aiming to predict their impact on the tendency to bugs. Results: We identified complexity-reducing interventions that reduce the probability of future bugs. Such interventions are relevant to 33\% of Python files and might reduce the tendency to bugs by 5.5 percentage points. Conclusions: We presented methods to evaluate the impact of interventions. The methods can identify a large number of natural interventions that are highly needed in causality research in many domains.

Problem

• static analysis
• alert removal
• code complexity
• bug proneness
• causality

Innovation

• alert removal
• randomized controlled trial
• labeling functions
• supervised learning
• causal inference