CursorCore: Assist Programming through Aligning Anything

2024-10-09

arXiv.org

AI Summary

Existing programming assistance tools exhibit low automation levels and struggle to integrate heterogeneous multimodal information—including user instructions, current code, and editing history—across diverse development scenarios. Method: This paper proposes a full-scenario conversational programming assistant framework, introducing the first APEval benchmark for systematic evaluation and the Programming-Instruct pipeline for automated high-quality instruction-data synthesis. It pioneers end-to-end modeling driven by multi-granularity information alignment, integrating LLM fine-tuning, multi-stage alignment, knowledge distillation from GitHub and Online Judge repositories, and a curated dataset of 219K high-quality samples. Contribution/Results: We release the CursorCore model series, which surpasses state-of-the-art baselines across equivalent parameter scales on code completion, insertion, and instruction-driven editing tasks. CursorCore supports seamless inline chat and automated editing in a unified interface. All code, models, and data are fully open-sourced.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large language models have been successfully applied to programming assistance tasks, such as code completion, code insertion, and instructional code editing. However, these applications remain insufficiently automated and struggle to effectively integrate various types of information during the programming process, including coding history, current code, and user instructions. In this work, we propose a new conversational framework that comprehensively integrates these information sources, collect data to train our models and evaluate their performance. Firstly, to thoroughly evaluate how well models align with different types of information and the quality of their outputs, we introduce a new benchmark, APEval (Assist Programming Eval), to comprehensively assess the performance of models in programming assistance tasks. Then, for data collection, we develop a data generation pipeline, Programming-Instruct, which synthesizes training data from diverse sources, such as GitHub and online judge platforms. This pipeline can automatically generate various types of messages throughout the programming process. Finally, using this pipeline, we generate 219K samples, fine-tune multiple models, and develop the CursorCore series. We show that CursorCore outperforms other models of comparable size. This framework unifies applications such as inline chat and automated editing, contributes to the advancement of coding assistants. Code, models and data are freely available at https://github.com/TechxGenus/CursorCore.

Problem

• Automating programming assistance with multi-source information integration
• Evaluating model alignment and output quality in coding tasks
• Generating diverse training data for enhanced programming support

Innovation

• Comprehensive conversational framework integrating multiple information sources
• New benchmark APEval for programming assistance evaluation
• Data generation pipeline Programming-Instruct synthesizing diverse training data