An Empirical Study of Security-Policy Related Issues in Open Source Projects

2025-10-07

AI Summary

Despite its intended role in standardizing vulnerability disclosure, the SECURITY.md file exhibits low adoption and ambiguous functionality within open-source projects. Method: To empirically assess its practical utility and operational challenges, we conducted a randomized sampling and content analysis of 1,200 GitHub issues related to six community health files—including SECURITY.md—quantitatively comparing issue resolution times and response counts. Contribution/Results: We find that 79.5% of SECURITY.md-related issues request file creation rather than report vulnerabilities; moreover, reports containing valid SECURITY.md links are resolved on average two days faster. These findings reveal that SECURITY.md currently functions primarily as a “security entry point” rather than an operational response guide—a novel empirical insight. The study provides actionable evidence and design implications for improving open-source security policies and enhancing vulnerability response efficiency.

Technology Category

• Application Domains: Security
• Search and Optimization: Evaluation and Analysis
• Natural Language Processing: Safety and Robustness

Application Category

• Security and Privacy: Large-scale security measurements
• Responsible Web: Measurement, analysis, and circumvention of Web censorship
• Web Mining and Content Analysis: Community question answering

Abstract

GitHub recommends that projects adopt a SECURITY.md file that outlines vulnerability reporting procedures. However, the effectiveness and operational challenges of such files are not yet fully understood. This study aims to clarify the challenges that SECURITY.md files face in the vulnerability reporting process within open-source communities. Specifically, we classified and analyzed the content of 711 randomly sampled issues related to SECURITY.md. We also conducted a quantitative comparative analysis of the close time and number of responses for issues concerning six community health files, including SECURITY.md. Our analysis revealed that 79.5% of SECURITY.md-related issues were requests to add the file, and reports that included links were closed, with a median time that was 2 days shorter. These findings offer practical insights for improving security reporting policies and community management, ultimately contributing to a more secure open-source ecosystem.

Problem

• Examining security policy implementation challenges in open source projects
• Analyzing effectiveness of SECURITY.md files in vulnerability reporting processes
• Identifying factors affecting issue resolution time for security policies

Innovation

• Analyzed 711 SECURITY.md issues for classification
• Compared response metrics across six health files
• Found link inclusion reduces issue resolution time