Enhancing Code Understanding for Impact Analysis by Combining Transformers and Program Dependence Graphs

2026-07-25

AI Summary

Traditional impact analysis approaches rely on heuristic coupling metrics, which are often hindered by fragility, high execution overhead, or dependence on extensive change histories, limiting their ability to accurately predict the scope of code changes. This work proposes Athena, the first method to integrate program dependence graphs with Transformer-based deep semantic code representations to enable automated impact analysis without requiring execution traces or historical data. We introduce Alexandria, a high-quality benchmark dataset comprising 25 open-source projects, and train and evaluate our model on fine-grained bug-fixing commits. Experimental results demonstrate that Athena significantly outperforms existing methods on Alexandria, achieving a mean reciprocal rank (mRR) of 60.32%, mean average precision (mAP) of 35.19%, and HIT@10 of 81.48%—representing performance gains of 9.55% to 11.68%.

Abstract

Impact analysis (IA) is a critical software maintenance task that identifies the effects of a given set of code changes on a larger software project with the intention of avoiding potential adverse effects. IA is a cognitively challenging task that involves reasoning about the abstract relationships between various code constructs. Given its difficulty, researchers have worked to automate IA with approaches that primarily use coupling metrics as a measure of the "connectedness" of different parts of a software project. Many of these coupling metrics rely on static, dynamic, or evolutionary information and are based on heuristics that tend to be brittle, require expensive execution analysis, or large histories of co-changes to accurately estimate impact sets. In this paper, we introduce a novel IA approach, called Athena, that combines a software system's dependence graph information with a conceptual coupling approach that uses advances in deep representation learning for code without the need for change histories and execution information. Previous IA benchmarks are small, containing fewer than ten software projects, and suffer from tangled commits, making it difficult to measure accurate results. Therefore, we constructed a large-scale IA benchmark, called Alexandria, from 25 open-source software projects, that utilizes fine-grained commit information from bug fixes. On this new benchmark, our best-performing approach configuration achieves mRR, mAP, and HIT@10 scores of 60.32%, 35.19%, and 81.48%, respectively. Through various ablations and qualitative analyses, we show that Athena's novel combination of program dependence graphs and conceptual coupling information leads it to outperform a simpler baseline by 10.34%, 9.55%, and 11.68% with statistical significance.

Problem

• Impact Analysis
• Code Understanding
• Program Dependence Graphs
• Conceptual Coupling
• Software Maintenance

Innovation

• Impact Analysis
• Program Dependence Graph
• Transformer
• Code Representation Learning
• Software Maintenance