A Study on the Impact of Fault localization Granularity for Repository-Scale Code Repair Tasks

2026-03-31

AI Summary

This study investigates the isolated impact of fault localization granularity on the effectiveness of automated program repair at the warehouse scale, assuming perfect localization accuracy. By systematically evaluating file-level, function-level, and line-level contextual information within a modified Agentless framework, we establish a reproducible experimental paradigm that decouples localization accuracy from repair performance for the first time on the SWE-Bench-Mini dataset. Our results demonstrate that function-level granularity achieves the highest overall repair rate; however, the optimal granularity varies across tasks, revealing a nuanced dependency between context granularity and the characteristics of individual repair tasks.

Technology Category

• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Search and Optimization: Local Search
• Multiagent Systems: Distributed Problem Solving

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Automatic program repair can be a challenging task, especially when resolving complex issues at a repository-level, which often involves issue reproduction, fault localization, code repair, testing and validation. Issues of this scale can be commonly found in popular GitHub repositories or datasets that are derived from them. Some repository-level approaches separate localization and repair into distinct phases. Where this is the case, the fault localization approaches vary in terms of the granularity of localization. Where the impact of granularity is explored to some degree for smaller datasets, not all isolate this issue from the separate question of localization accuracy by testing code repair under the assumption of perfect fault localization. To the best of the authors' knowledge, no repository-scale studies have explicitly investigated granularity under this assumption, nor conducted a systematic empirical comparison of granularity levels in isolation. We propose a framework for performing such tests by modifying the localization phase of the Agentless framework to retrieve ground-truth localization data and include this as context in the prompt fed to the repair phase. We show that under this configuration and as a generalization over the SWE-Bench-Mini dataset, function-level granularity yields the highest repair rate against line-level and file-level. However, a deeper dive suggests that the ideal granularity may in fact be task dependent. This study is not intended to improve on the state-of-the-art, nor do we intend for results to be compared against any complete agentic frameworks. Rather, we present a proof of concept for investigating how fault localization may impact automatic code repair in repository-scale scenarios. We present preliminary findings to this end and encourage further research into this relationship between the two phases.

Problem

• fault localization granularity
• repository-scale code repair
• automatic program repair
• localization accuracy
• code repair performance

Innovation

• fault localization granularity
• repository-scale program repair
• automatic program repair
• empirical study
• prompt-based repair