🤖 AI Summary
This study investigates the differential impact of large language models (LLMs) on capability innovation (i.e., exploring novel functionalities) versus iterative innovation (i.e., optimizing and maintaining existing code) in open-source collaborative development. Leveraging GitHub Copilot’s phased, language-specific rollout as a quasi-natural experiment, we employ a multi-period difference-in-differences design, using variation in programming language support as an exogenous shock to identify causal LLM effects. Our key contribution is the first empirical evidence that—under unguided, spontaneous collaboration—LLMs significantly boost iterative innovation while exerting limited influence on capability innovation. This effect strengthens with model upgrades (e.g., the 2022 release) and higher project activity, and is especially pronounced in Python and Rust projects. The findings indicate that current LLMs are better aligned with maintenance-oriented development than with exploratory feature creation, offering critical empirical insights into the pathways and boundaries of AI-augmented open-source innovation.
📝 Abstract
Large Language Models (LLMs) have been shown to enhance individual productivity in guided settings. Whereas LLMs are likely to also transform innovation processes in a collaborative work setting, it is unclear what trajectory this transformation will follow. Innovation in these contexts encompasses both capability innovation that explores new possibilities by acquiring new competencies in a project and iterative innovation that exploits existing foundations by enhancing established competencies and improving project quality. Whether LLMs affect these two aspects of collaborative work and to what extent is an open empirical question. Open-source development provides an ideal setting to examine LLM impacts on these innovation types, as its voluntary and open/collaborative nature of contributions provides the greatest opportunity for technological augmentation. We focus on open-source projects on GitHub by leveraging a natural experiment around the selective rollout of GitHub Copilot (a programming-focused LLM) in October 2021, where GitHub Copilot selectively supported programming languages like Python or Rust, but not R or Haskell. We observe a significant jump in overall contributions, suggesting that LLMs effectively augment collaborative innovation in an unguided setting. Interestingly, Copilot's launch increased iterative innovation focused on maintenance-related or feature-refining contributions significantly more than it did capability innovation through code-development or feature-introducing commits. This disparity was more pronounced after the model upgrade in June 2022 and was evident in active projects with extensive coding activity, suggesting that as both LLM capabilities and/or available contextual information improve, the gap between capability and iterative innovation may widen. We discuss practical and policy implications to incentivize high-value innovative solutions.