Score
Programmatic use of the GitHub API and related tooling to identify, collect, clean, and link repository- and user-level artifacts (e.g., CI/CD configs) at scale across large corpora while preserving provenance and temporal consistency.
This study addresses the lack of systematic understanding regarding the content and long-term evolution of GitHub repositories. It presents the first large-scale empirical analysis of 10,000 real-world open-source repositories, combining static content parsing with time-series modeling to trace the evolution of files, directories, and file extensions over the past decade. The findings reveal that README.md, .gitignore, and LICENSE have become standard components; CI/CD tooling has shifted from diversity toward dominance by GitHub Actions; configuration formats exhibit a clear rise of TOML, YAML, and JSON alongside the decline of XML; and Dockerfiles as well as LLM-related files (e.g., AGENTS.md) have grown significantly. This work provides quantitative evidence for understanding technological shifts and standardization processes in the open-source ecosystem.
Despite the widespread use of dotfiles for personal configuration management (e.g., vim, bash/zsh), empirical understanding of how developers share and maintain them remains scarce. Method: This study conducts the first large-scale empirical analysis of publicly available dotfiles repositories on GitHub, leveraging GitHub API-collected metadata and textual analysis, complemented by statistical modeling to quantify configuration type distributions, update frequencies, change motivations, and maintenance behavior patterns. Contribution/Results: We find that 25.8% of top GitHub users publicly maintain dotfiles; editor and shell configurations dominate (>80%); 63.3% of updates stem from personal customization rather than application dependencies; and update frequencies exhibit substantial inter-user variability—reflecting highly individualized configuration practices. These findings fill a critical gap in empirical software configuration research and provide data-driven insights to inform the design of configuration management tools and infrastructure.
This study addresses the lack of systematic understanding regarding the application domains, maintenance characteristics, and effective design practices of GitHub template repositories. Conducting the first large-scale empirical investigation, the work integrates data mining, statistical analysis, code quality assessment tools—detecting code smells, vulnerabilities, and security hotspots—and an LLM-as-a-judge classification approach to systematically uncover domain distributions, language-specific quality variations, and maintenance patterns. The findings reveal web development as the dominant application domain, with high-quality templates consistently adhering to software engineering best practices and offering comprehensive documentation. Through qualitative evaluation, the study distills actionable design guidelines and identifies common pitfalls, providing practical guidance for developers creating or using template repositories.
This study addresses the critical threat posed by GitHub abuse to software supply chain security, an area lacking systematic investigation and effective detection methods. To bridge this gap, the work proposes the first comprehensive taxonomy of GitHub abuse behaviors tailored for software security, encompassing both observable symptoms and underlying root causes. The authors construct a manually annotated dataset based on 392 publicly reported cases and develop a unified, cross-repository and cross-user account detection framework capable of identifying multiple abuse categories. Experimental evaluation demonstrates that the proposed framework achieves consistently high performance across all abuse types, with F1 scores exceeding 89%, substantially advancing detection capabilities and filling a significant void in the existing research landscape.
This study addresses the observability challenge in maintaining critical libraries within open-source ecosystems by systematically assessing the accessibility of GitHub repository URLs associated with PyPI and npm packages. We propose a dual-perspective analytical framework integrating direct dependency relationships with PageRank-based importance scoring, enabling the first quantitative assessment of repository URL coverage and failure cause distributions across both ecosystems. Key findings include: (1) counterintuitively, higher-importance packages exhibit greater URL accessibility; (2) per-package URL accessibility rates are 73.8% (PyPI) and 69.4% (npm), improving to 80.1% and 81.1%, respectively, when considering transitive dependency chains; and (3) “missing repository URL configuration” is the predominant failure cause—accounting for 17.9% of PyPI and 39.6% of npm cases. These results provide empirical foundations and methodological support for enhancing supply-chain security monitoring in open-source software.
This study addresses the lack of systematic understanding in the configuration and maintenance of CI/CD caching, which imposes a significant burden on developers despite its benefits for build efficiency. Through a large-scale empirical analysis of 952 repositories on GitHub Actions—encompassing 1,556 workflow files and over ten thousand cache-related changes—the authors employ code mining, configuration analysis, commit tracing, and statistical modeling to uncover real-world caching practices, evolutionary patterns, and human-bot collaboration in maintenance. The findings reveal that cache adopters are more active, caching strategies are diverse and frequently adjusted, and build- and test-related tasks evolve rapidly. Manual interventions primarily address misconfigurations, whereas version upgrades are predominantly automated by bots. The work quantifies the maintenance overhead of caching and provides empirical foundations for improving developer tooling.
This study addresses the lack of systematic understanding regarding the implementation and maintenance of the Model Context Protocol (MCP) in real-world open-source projects. To bridge this gap, we introduce a transparent, reproducible multi-stage validation pipeline that integrates GitHub REST/GraphQL APIs with custom Python scripts to systematically annotate structural evidence, classify repository roles, and filter out non-functional examples from 3,238 candidate repositories. This process yields a high-quality dataset of 2,297 verified MCP projects, achieving a validation precision of 83% at 95% confidence. Our analysis reveals Python and TypeScript as the dominant implementation languages and identifies hybrid architecture as the most prevalent design pattern, thereby establishing the first large-scale empirical benchmark for MCP ecosystem research.
This study addresses the lack of systematic understanding regarding how GitHub Actions workflows are used in real-world scenarios, how developers respond to workflow failures, and how these practices relate to project characteristics. Combining large-scale quantitative analysis of 258,300 workflow runs with qualitative case studies across 21 diverse repositories, this work identifies three typical patterns developers employ to handle workflow failures and uncovers a “configuration–usage gap”—where YAML configurations exist but workflows remain effectively unused. Furthermore, the study empirically validates five hypotheses linking project features to workflow usage intensity, revealing a significant positive correlation between high usage intensity and low failure rates. These findings provide actionable empirical evidence for improving CI/CD practices.
This work addresses persistent challenges faced by coding agents in evolving code repositories—namely redundant retrieval, fragmented context, and opaque computational costs. To overcome these issues, the authors propose a multi-view context service that constructs reusable lexical, dense, and structured views for each commit, unifying support for code search, symbol navigation, and bounded context provision. By explicitly defining operation-level validity boundaries, the system enables cross-edit view persistence and efficient incremental updates. Experimental results demonstrate that, compared to full reconstruction, graph and vector index updates achieve speedups of 8.7× and 25.4×, respectively; static navigation latency is reduced to one-fourth of real-time serving requirements; and context-aware strategies cut trajectory token consumption by 50%–87%.
This work addresses the limitations of existing reproducibility assessment methods, which rely on manual annotations and thus lack scalability and authentic supervision signals reflecting real-world reproduction challenges. The authors propose the first scalable evaluation framework that leverages GitHub user-submitted issues as natural supervision, enabling large-scale assessment of large language model (LLM) agents’ ability to identify paper-to-code reproducibility issues without human annotation. By integrating language understanding with code context analysis, the approach enables non-execution-based detection of reproducibility barriers. Experimental results demonstrate that the best-performing LLM agent identifies at least one semantically relevant reproducibility issue—aligned with those reported by humans—in approximately 90% of the evaluated papers, exhibiting strong performance in both failure detection and semantic localization.