An approach for API synthesis using large language models

2025-02-21

AI Summary

To address the inefficiency and poor generalization of traditional search strategies in API synthesis under large-scale implementation spaces, this paper introduces, for the first time, a systematic large language model (LLM)-driven approach to component-based API synthesis. We propose an end-to-end method leveraging prompt engineering, in-context learning, and code generation to replace exhaustive search. Evaluated on a realistic programming task benchmark comprising 135 tasks, our method successfully synthesizes correct APIs for 133 tasks—achieving a 98.5% accuracy rate—significantly outperforming the state-of-the-art tool FrAngel. Our key contributions are: (1) establishing the first LLM-driven paradigm specifically designed for API synthesis; (2) empirically demonstrating LLMs’ capability to jointly model developer intent and code patterns; and (3) substantially improving both synthesis efficiency and cross-task generalization performance.

Technology Category

• Natural Language Processing: Code Generation / Program Synthesis from Natural Language
• Machine Learning: Large Multimodal Models (LMMs)
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• Search and Retrieval-Augmented AI: Large language models for search
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

APIs play a pivotal role in modern software development by enabling seamless communication and integration between various systems, applications, and services. Component-based API synthesis is a form of program synthesis that constructs an API by assembling predefined components from a library. Existing API synthesis techniques typically implement dedicated search strategies over bounded spaces of possible implementations, which can be very large and time consuming to explore. In this paper, we present a novel approach of using large language models (LLMs) in API synthesis. LLMs offer a foundational technology to capture developer insights and provide an ideal framework for enabling more effective API synthesis. We perform an experimental evaluation of our approach using 135 real-world programming tasks, and compare it with FrAngel, a state-of-the-art API synthesis tool. The experimental results show that our approach completes 133 of the tasks, and overall outperforms FrAngel. We believe LLMs provide a very useful foundation for tackling the problem of API synthesis, in particular, and program synthesis, in general.

Problem

• Enhancing API synthesis with large language models.
• Addressing inefficiency in existing API synthesis techniques.
• Improving program synthesis through developer insights capture.

Innovation

• Uses large language models
• Enhances API synthesis efficiency
• Outperforms state-of-the-art tool