manual code review

Careful human inspection of source or decompiled code and related artifacts to identify root causes of failures, complementary evidence across views, and systematic error patterns that automated tools miss, enabling reproducibility analysis and bug triage.

manualcodereview

12-Month Skill Trend

Momentum and market value over time
Trending
Score
+20 in 12 mo
96
12 mo agoNow
Career
Value
+$12K in 12 mo
$42K/year
12 mo agoNow

Recommended Survey Paper

Quick overview of the field
View more

Must-Read Papers

Most classic and influential ideas
View more

This study addresses the longstanding fragmentation in software artifact traceability research, characterized by incomplete linkages, ambiguous techniques, and disconnected application contexts. Through a systematic literature review, it constructs the first comprehensive traceability landscape encompassing 22 artifact types and 23 relationship kinds, and introduces a technology decision map, a standardized evaluation benchmark, and a role-oriented dynamic path alignment framework. The work uncovers critical challenges: a pervasive code-centric bias, a reproducibility crisis stemming from only 37% of studies releasing open-source artifacts, and a significant adoption gap with 95% of proposed tools never deployed in industry. In response, it offers targeted strategies to bridge these gaps, establishing a unified knowledge foundation for future research and practical implementation in traceability.

artifact associationssoftware artifactssoftware traceability

Existing bug report datasets suffer from narrow coverage, poor timeliness, and incomplete metadata, hindering the application of machine learning in software quality analysis. To address these limitations, we introduce the first modern, cross-platform (GitHub/Bugzilla/Jira), cross-project (nine active open-source projects) bug report benchmark dataset, comprising over 150,000 standardized reports with comprehensive metadata and pre-split train/test splits. Our methodology includes a unified schema for semantic modeling, structured field annotation, multi-source heterogeneous data cleaning, and a Jupyter-based exploratory analysis framework. The dataset enables rigorous benchmarking for tasks including duplicate detection, RAG-enhanced generation, and automated triage—demonstrating empirically improved accuracy and relevance. Since its open release, it has become a mainstream benchmark resource for intelligent software defect analysis.

Insufficient metadata for machine learning in bug report analysisLimited scope and outdated content in existing bug report datasetsNeed for standardized datasets to support diverse SE research tasks

The unclear spatiotemporal distribution patterns of defects in multi-fault repositories hinder cost-effective maintenance optimization. Method: We conduct an empirical study on 16 Java/Python open-source projects from Defects4J and BugsInPy, analyzing their multi-fault versions via version history tracing and precise fault localization. Contribution/Results: Our analysis reveals—temporally—that long-standing unpatched defects commonly coexist across versions, and—spatially—that defect distributions exhibit low concentration (few hotspots) and high uniformity. This challenges the conventional single-fault assumption and provides the first systematic empirical validation of widespread multi-fault coexistence and non-localized defect clustering. The findings establish a more realistic, scalable empirical foundation for test case prioritization, repair resource allocation, and evaluation of tools in multi-defect scenarios, supported by rigorously curated data.

Analyzes temporal and spatial characteristics of multi-fault systemsChallenges single-fault assumptions in Defects4J and BugsInPy datasetsInvestigates distribution and longevity of faults in Java/Python projects

Software source code often harbours"hotspots": small portions of the code that change far more often than the rest of the project and thus concentrate maintenance activity. We mine the complete version histories of 91 evolving, actively developed GitHub repositories and identify 15 recurring line-level hotspot patterns that explain why these hotspots emerge. The three most prevalent patterns are Pinned Version Bump (26%), revealing brittle release practices; Long Line Change (17%), signalling deficient layout; and Formatting Ping-Pong (9%), indicating missing or inconsistent style automation. Surprisingly, automated accounts generate 74% of all hotspot edits, suggesting that bot activity is a dominant but largely avoidable source of noise in change histories. By mapping each pattern to concrete refactoring guidelines and continuous integration checks, our taxonomy equips practitioners with actionable steps to curb hotspots and systematically improve software quality in terms of configurability, stability, and changeability.

change historycode churnmaintenance activity

This study addresses the prevalent yet elusive issue of “silent failures” in AI-generated code—functional degradations or errors that appear syntactically correct but evade detection by conventional testing methods. The authors propose the Reward Shaping Failure Hypothesis, attributing this phenomenon to optimization dynamics in human feedback alignment, and introduce the novel concept of “failure truthfulness.” Building on this insight, they develop AIRA, the first targeted risk auditing framework for AI-generated code, which employs structured static analysis with 15 deterministic rules across JavaScript, Python, and TypeScript, with particular emphasis on exception handling. Empirical evaluation reveals that AI-generated code exhibits 1.80 times more high-severity issues per file (0.435) than human-written code (0.242), a consistent effect across languages, with deficiencies in exception handling being especially pronounced.

AI-generated codecode reliabilityfail-soft behavior

Latest Papers

What's happening recently
View more

This study addresses the persistent occurrence of software defects after release, particularly in C/C++ and Java systems, whose underlying causes remain poorly understood. Through a large-scale empirical analysis of over 14,000 open-source projects, the work systematically compares pre-release and post-release defect characteristics using multidimensional metrics—including code complexity, size, change frequency, and development history—and employs statistical modeling to uncover key patterns. It reveals for the first time that post-release defects are significantly concentrated in legacy modules that undergo frequent modifications, with their root causes primarily stemming from dynamic evolutionary pressures rather than static code structure. Furthermore, such defects exhibit longer repair cycles and higher complexity, offering empirical grounding for targeted testing strategies and improved reliability assurance.

defect characterizationpost-release defectsresidual faults

This work addresses the limited trust developers place in AI-generated bug reports due to their frequent lack of actionability and reproducibility. The authors propose a novel approach that integrates code coverage analysis with large language models (LLMs) to automatically detect defects in uncovered code regions and generate structured bug reports containing severity ratings, reproduction steps, and repair suggestions. A key innovation is an LLM-driven prioritization mechanism that substantially outperforms traditional rule-based methods. Evaluated on 13 Python projects, the method produced 10,467 reports; manual assessment of the top 130 revealed an 84.6% validity rate. Compared to CoverUp, it achieves higher defect validity (81.0% vs. 76.2%), a 50% improvement in P@3, and a 41% gain in mean reciprocal rank (MRR).

actionabilityAI-generated issue reportsautomated bug detection

This study systematically evaluates whether large language models can rediscover six real-world, system-level vulnerabilities disclosed by the Anthropic Mythos project using only read-only source code, with critical metadata such as CVE identifiers explicitly withheld. Experiments were conducted with GPT-5.5 xhigh, Claude Opus 4.7, and Kimi K2 under a unified matching scoring protocol and replicated trial design, targeting file-level vulnerability rediscovery. Results show that GPT-5.5 xhigh successfully reproduced five instances (spanning two distinct tasks), Claude Opus 4.7 succeeded once, and Kimi K2 achieved no successes. The primary cause of failure across models was premature fixation on plausible but incorrect code locations. This work represents the first systematic assessment of large language models’ precise vulnerability localization capabilities in complex, real-world scenarios without reliance on key prompting cues.

bug rediscoverycontrolled benchmarkinglarge language models

This work addresses the reliability of automatic program repair (APR) evaluation by proposing the first rigorous reproducibility criteria for APR-oriented defect datasets. Applying these standards—encompassing automated test execution, static analysis, patch behavior comparison, and test suite adequacy checks—to the widely used Defects4J benchmark reveals significant shortcomings: 21.6% of its defects are unsuitable for APR evaluation, and an additional 7.1% suffer from substantially inadequate test suites, collectively rendering 28.7% (239 defects) prone to unreliable assessment. To promote more rigorous future research, the authors release the first open-source Java APR evaluation framework and call on the community to prioritize benchmark quality in empirical studies.

automated program repairbenchmark datasetDefects4J

This study addresses the lack of empirical analysis on how quality defects in requirements documents affect the performance of automated trace link recovery (TLR). For the first time, it systematically annotates 28 types of requirements quality defects across 189 use cases from two datasets, evaluates five state-of-the-art TLR methods, and analyzes their performance impacts using both statistical significance and effect size measures. The findings reveal that specific defect types differentially influence TLR effectiveness: while certain defects significantly degrade performance, others unexpectedly enhance it. Beyond identifying key factors that either hinder or facilitate TLR accuracy, this work demonstrates that the choice of TLR method should be strategically tailored to the quality characteristics of the underlying requirements documentation.

empirical studyquality defectsrequirements quality

Hot Scholars

CT

Christoph Treude

Associate Professor of Computer Science, Singapore Management University
Software EngineeringEmpirical Software EngineeringHuman-AI InteractionAI for Science
AT

Amjed Tahir

Massey University
AI4SESoftware TestingEmpirical Software Engineering
PL

Peng Liang

School of Computer Science, Wuhan University
Software EngineeringSoftware ArchitectureEmpirical Software Engineering
CF

Chunrong Fang

Software Institute, Nanjing University
Software TestingSoftware EngineeringComputer Science
WS

Weisong Sun

Nanyang Technological University
Trustworthy Intelligent SE (Software Engineering)