GitBugs: Bug Reports for Duplicate Detection, Retrieval Augmented Generation, Triage, and More

2025-04-13

AI Summary

Existing bug report datasets suffer from narrow coverage, poor timeliness, and incomplete metadata, hindering the application of machine learning in software quality analysis. To address these limitations, we introduce the first modern, cross-platform (GitHub/Bugzilla/Jira), cross-project (nine active open-source projects) bug report benchmark dataset, comprising over 150,000 standardized reports with comprehensive metadata and pre-split train/test splits. Our methodology includes a unified schema for semantic modeling, structured field annotation, multi-source heterogeneous data cleaning, and a Jupyter-based exploratory analysis framework. The dataset enables rigorous benchmarking for tasks including duplicate detection, RAG-enhanced generation, and automated triage—demonstrating empirically improved accuracy and relevance. Since its open release, it has become a mainstream benchmark resource for intelligent software defect analysis.

Technology Category

• Data Mining & Knowledge Management: Representing, Reasoning, and Using Provenance, Trust
• Knowledge Representation and Reasoning: Diagnosis and Abductive Reasoning
• Natural Language Processing: Safety and Robustness

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Web Mining and Content Analysis: Web data integration and cleaning

Abstract

Bug reports provide critical insights into software quality, yet existing datasets often suffer from limited scope, outdated content, or insufficient metadata for machine learning. To address these limitations, we present GitBugs-a comprehen- sive and up-to-date dataset comprising over 150,000 bug reports from nine actively maintained open-source projects, including Firefox, Cassandra, and VS Code. GitBugs aggregates data from Github, Bugzilla and Jira issue trackers, offering standardized categorical fields for classification tasks and predefined train/test splits for duplicate bug detection. In addition, it includes ex- ploratory analysis notebooks and detailed project-level statistics, such as duplicate rates and resolution times. GitBugs supports various software engineering research tasks, including duplicate detection, retrieval augmented generation, resolution prediction, automated triaging, and temporal analysis. The openly licensed dataset provides a valuable cross-project resource for bench- marking and advancing automated bug report analysis. Access the data and code at https://github.com/av9ash/gitbugs/.

Problem

• Limited scope and outdated content in existing bug report datasets
• Insufficient metadata for machine learning in bug report analysis
• Need for standardized datasets to support diverse SE research tasks

Innovation

• Comprehensive dataset from multiple issue trackers
• Standardized fields for classification tasks
• Supports diverse bug report analysis tasks