Full Line Code Completion: Bringing AI to Desktop

2024-05-14

arXiv.org

AI Summary

Existing code completion methods predominantly rely on cloud-based inference, raising concerns regarding data privacy leakage, network latency, and delayed response—especially for multi-token (full-line) completions, where efficient and secure on-device solutions remain scarce. This paper introduces the first production-grade, fully on-device, full-line multi-token code completion system deeply integrated into the JetBrains IntelliJ platform. It operates entirely locally using a lightweight Transformer model, local index-based retrieval, and syntax-aware decoding—ensuring low latency, minimal memory footprint (<500 MB RAM), and strong data privacy under stringent resource constraints. We establish the first design principles and industrial resource benchmarks for on-device code completion, bridging the gap between academic research and real-world deployment. Dual-mode (offline/online) evaluation shows a 1.3× improvement in Python completion adoption rate. The system is pre-installed across all JetBrains IDEs, serving over one million developers worldwide.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Learning on the Edge & Model Compression
• Cognitive Modeling & Cognitive Systems: Neural Spike Coding

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

In recent years, several industrial solutions for the problem of multi-token code completion appeared, each making a great advance in the area but mostly focusing on cloud-based runtime and avoiding working on the end user's device. In this work, we describe our approach for building a multi-token code completion feature for the JetBrains' IntelliJ Platform, which we call Full Line Code Completion. The feature suggests only syntactically correct code and works fully locally, i.e., data querying and the generation of suggestions happens on the end user's machine. We share important time and memory-consumption restrictions, as well as design principles that a code completion engine should satisfy. Working entirely on the end user's device, our code completion engine enriches user experience while being not only fast and compact but also secure. We share a number of useful techniques to meet the stated development constraints and also describe offline and online evaluation pipelines that allowed us to make better decisions. Our online evaluation shows that the usage of the tool leads to 1.3 times more Python code in the IDE being produced by code completion. The described solution was initially started with a help of researchers and was then bundled into all JetBrains IDEs where it is now used by millions of users. Thus, we believe that this work is useful for bridging academia and industry, providing researchers with the knowledge of what happens when complex research-based solutions are integrated into real products.

Problem

• Code Completion
• Privacy Security
• Efficiency

Innovation

• Full Line Code Completion
• Syntax Correctness and Usability
• Empirical Study Validation