Promptware Engineering: Software Engineering for LLM Prompt Development

2025-03-04

AI Summary

To address the “prompting software crisis” arising from the lack of systematic methodologies for prompt development, this paper proposes “Prompt Software Engineering” (PSE)—a novel paradigm that adapts software engineering principles to the full lifecycle management of natural language prompts in non-deterministic large language model (LLM) environments. Methodologically, we introduce, for the first time, a comprehensive framework tailored to LLM characteristics, encompassing prompt requirement analysis, formal design, repeatable testing, interpretable debugging, and iterative evolution—integrating semantic modeling, context-aware testing, and evolutionary mechanisms. Our core contribution lies in extending classical software engineering beyond its traditional boundaries of determinism and precision, thereby establishing the first end-to-end engineering methodology and research roadmap for prompting. This advances LLM-native application development by enabling reusable, verifiable, and maintainable prompt-based systems.

Technology Category

Natural Language Processing: Prompt Engineering / PromptingMachine Learning: Large Multimodal Models (LMMs)Search and Optimization: Sampling/Simulation-based Search

Application Category

User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendationSearch and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved informationSemantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

Large Language Models (LLMs) are increasingly integrated into software applications, with prompts serving as the primary 'programming' interface to guide their behavior. As a result, a new software paradigm, promptware, has emerged, using natural language prompts to interact with LLMs and enabling complex tasks without traditional coding. Unlike traditional software, which relies on formal programming languages and deterministic runtime environments, promptware is based on ambiguous, unstructured, and context-dependent natural language and operates on LLMs as runtime environments, which are probabilistic and non-deterministic. These fundamental differences introduce unique challenges in prompt development. In practice, prompt development is largely ad hoc and experimental, relying on a time-consuming trial-and-error process - a challenge we term the 'promptware crisis.' To address this, we propose promptware engineering, a new methodology that adapts established software engineering principles to the process of prompt development. Building on decades of success in traditional software engineering, we envision a systematic framework that includes prompt requirements engineering, design, implementation, testing, debugging, and evolution. Unlike traditional software engineering, our framework is specifically tailored to the unique characteristics of prompt development. This paper outlines a comprehensive roadmap for promptware engineering, identifying key research directions and offering actionable insights to advance LLM-based software development.

Problem

• Addresses challenges in LLM prompt development.
• Proposes promptware engineering for systematic prompt development.
• Adapts software engineering principles to LLM-based software.

Innovation

• Adapts software engineering principles to prompt development
• Systematic framework for prompt requirements and design
• Tailored methodology for probabilistic LLM environments