Agentic Coding in the Wild: Characterizing GitHub Copilot Traces at Production Scale

2026-07-30

AI Summary

This work addresses the gap in existing large model serving systems, which lack an understanding of the real-world workload characteristics of AI programming agents and thus struggle to efficiently support their interleaved multi-step reasoning and tool invocation patterns. Leveraging a large-scale dataset from GitHub Copilot—comprising 3.2 million users, 13 million sessions, and 95 trillion tokens sampled in June 2026—the study provides the first systematic characterization of agent sessions, revealing key traits such as highly sparse user interactions and intra-turn KV cache hit rates as high as 90%, which drop sharply to 55% across turns. Through extensive log analysis, KV cache modeling, and a lightweight idle-time prediction algorithm, the proposed predictor captures 86–90% of total idle time, offering empirical insights and design principles for building agent-native infrastructure.

Abstract

AI coding agents like GitHub Copilot, Claude Code, and Codex interleave multi-step LLM inference with tool execution, creating a workload different from chatbots. We present the first production-scale characterization of this workload using sampled GitHub Copilot traces from June 2026, comprising 3.2M users, 13M sessions, 761M LLM calls, and 95T tokens. Our analysis reveals distinctive workload properties with important systems implications. For example, agentic coding sessions consist of sparse user-initiated turns, each unfolding into an autonomous agent loop of LLM calls almost always coupled with tool execution. This structure yields KV cache hit rates averaging 90% within a turn, but falling to 55\% across turn boundaries and drastically invalidated after events like model switches or context compaction. Diverse workflows and user behaviors are observed with variable and long-tailed token consumption, time span, and tool calls. We highlight the difference between quick agentic turnaround times and the minutes-long user idle periods at turn boundaries, and design a lightweight idle-time predictor that captures 86-90\% of total idle time, enabling proactive decisions for efficient resource orchestration. These findings challenge assumptions underlying current LLM-serving systems and provide an empirical foundation for agent-native infrastructure.

Problem

• agentic coding
• LLM serving
• workload characterization
• production-scale
• KV cache

Innovation

• agentic coding
• LLM inference
• KV cache efficiency
• idle-time prediction
• production-scale characterization