Problem
Research questions and friction points this paper is trying to address.
defect prediction
bug-fixing commits
labeling
data-quality problems
Labelling Bug-Fixing Commits with Local Open-Weight Language Models
🤖 AI Summary
本文探讨了使用本地运行的开放权重语言模型从提交内容中自动标记修复bug的提交，以解决现有标签方法中的噪声问题。
📝 Abstract
Defect prediction depends on knowing which commits fix bugs, yet the labels that encode this are produced by routes that each introduce noise. Reused benchmarks carry documented data-quality problems, issue-tracker links are biased and the underlying reports are frequently mistyped, and matching keywords in commit messages is a coarse heuristic. This paper examines whether commits can be labelled as bug fixes from their content alone, using open-weight language models that run locally and therefore keep the process reproducible, inexpensive at corpus scale, usable on proprietary code, and independent of any issue tracker. Against datasets of manually validated and curated bug fixes spanning Java, Python, and JavaScript, we compare a keyword baseline with a set of open-weight models of varying size, prompting each with the commit message and the code diff. On the manually validated corpus the keyword baseline recovers fewer than half of the fixes, whereas the open-weight models recover the large majority and outperform it repository by repository with statistical significance, and larger models do not consistently outperform smaller ones. We further show that evaluation corpora without negative examples cannot support a precision-aware comparison of such classifiers. We release the labelling pipeline together with a labelled, multi-language corpus produced by the recommended configuration, as a reproducible silver-standard resource for building current, project-specific datasets.
Innovation
Methods, ideas, or system contributions that make the work stand out.
open-weight language models
local execution
reproducibility
commit labeling
🔎 Similar Papers
No similar papers found.
