AgenTag: Attribution of AI Coding Agents from Behavioral Fingerprints

2026-08-01

AI Summary

This study addresses the challenge of attributing code contributions to AI coding agents when they operate under developer accounts, which obscures authorship and undermines repository governance and empirical software research. To tackle this, the authors propose AgenTag, a novel framework that enables AI agent provenance tracing in open-world settings. AgenTag integrates multimodal features—including pull request descriptions, commit messages, and code diffs—and leverages supervised contrastive learning with classification to support few-shot registration of new agents. The findings reveal that agent-identifiable stylistic signatures primarily stem from communicative behaviors, such as change descriptions, rather than the generated code itself. Evaluated on the AIDev dataset, the model achieves a weighted F1 score of 0.96, a balanced F1 of 0.89 for distinguishing AI from human contributions, and an AUC of 0.84 for detecting previously unseen agents.

Abstract

AI coding agents increasingly author pull requests (PRs), often under developers' own accounts, obscuring who actually produced a change. Reliable attribution is important for repository governance, empirical studies of AI-assisted software development, and measuring the impact of AI coding agents. Existing work focuses on closed-set identification of known agents, leaving the practical limits of open-world AI coding agent attribution largely unexplored. In this paper, we present AgenTag, a multimodal framework for open-world AI coding agent attribution, evaluated on AIDev, comprising 33,580 PRs from five AI coding agents and 6,618 human-authored PRs. We represent each PR using textual, behavioral, and code-based modalities, and compare conventional classification with supervised contrastive learning for open-set recognition and few-shot enrollment of previously unseen agents. AgenTag identifies authoring agents with a weighted F1 of 0.96 (macro F1 of 0.84), distinguishes AI- from human-authored PRs with a balanced F1 of 0.89, and detects previously unseen agents with an AUC of 0.84. We further show that PR descriptions and commit messages provide nearly all of the attribution signal, whereas code diffs contribute little across multiple representations, indicating that coding agents are distinguished primarily by how they communicate changes rather than by the code they generate. Moreover, these behavioral fingerprints persist after removing explicit self-disclosed markers, demonstrating that attribution relies largely on latent stylistic characteristics. These findings show that reliable attribution of AI coding agents is feasible and clarify the practical trade-offs between attribution accuracy and the information required to achieve it.

Problem

• AI coding agents
• attribution
• open-world identification
• behavioral fingerprints
• software development

Innovation

• AI coding agent attribution
• behavioral fingerprints
• open-world recognition
• supervised contrastive learning
• multimodal representation