Who Said CVE? How Vulnerability Identifiers Are Mentioned by Humans, Bots, and Agents in Pull Requests

2026-01-27

AI Summary

This study addresses the lack of systematic understanding regarding how vulnerability identifiers—such as CVE, CWE, and GHSA—are used and referenced by humans, bots, and AI agents in real-world software development. For the first time, it systematically compares the behavioral patterns of these three actor types in referencing vulnerability identifiers within GitHub pull requests, leveraging the AIDevPop dataset augmented with an expanded set of PRs. Through a combination of quantitative analysis and qualitative content examination, the research reveals that bots account for 69.1% of all mentions, primarily in the context of dependency updates, whereas humans and AI agents, despite fewer references, engage with identifiers in richer contextual settings—particularly in vulnerability remediation and technical discussions—highlighting distinct roles and collaborative potential among different actors in security practices.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Human-Robot Interaction
• Cognitive Modeling & Cognitive Systems: Simulating Human Behavior

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Vulnerability identifiers such as CVE, CWE, and GHSA are standardised references to known software security issues, yet their use in practice is not well understood. This paper compares vulnerability ID use in GitHub pull requests authored by autonomous agents, bots, and human developers. Using the AIDev pop dataset and an augmented set of pull requests from the same repositories, we analyse who mentions vulnerability identifiers and where they appear. Bots account for around 69.1% of all mentions, usually adding few identifiers in pull request descriptions, while human and agent mentions are rarer but span more locations. Qualitative analysis shows that bots mainly reference identifiers in automated dependency updates and audits, whereas humans and agents use them to support fixes, maintenance, and discussion.

Problem

• vulnerability identifiers
• CVE
• pull requests
• bots
• software security

Innovation

• vulnerability identifiers
• pull requests
• autonomous agents
• software security
• empirical study