🤖 AI Summary
This work addresses the challenge of generating reliable system models directly from natural language requirements by proposing a novel framework that leverages large language models (LLMs) for the automatic generation and repair of finite state machines (FSMs). It introduces, for the first time, the use of LLMs in FSM construction and innovatively integrates FSM mutation analysis with automated test generation to establish an expert feedback–driven optimization mechanism. Experimental validation on synthetic datasets using GPT-4 demonstrates the effectiveness of the approach, significantly improving the correctness and completeness of the generated FSMs. The study thus establishes a new paradigm for model-driven engineering that synergistically combines generative AI with formal verification techniques.
📝 Abstract
Finite state machines (FSM) are executable formal specifications of reactive systems. These machines are designed based on systems' requirements. The requirements are often recorded in textual documents written in natural languages. FSMs play a crucial role in different phases of the model-driven system engineering (MDE). For example, they serve to automate testing activities. FSM quality is critical: the lower the quality of FSM, the higher the number of faults surviving the testing phase and the higher the risk of failure of the systems in production, which could lead to catastrophic scenarios. Therefore, this paper leverages recent advances in the domain of LLM to propose an LLM-based framework for designing FSMs from requirements. The framework also suggests an expert-centric approach based on FSM mutation and test generation for repairing the FSMs produced by LLMs. This paper also provides an experimental analysis and evaluation of LLM's capacities in performing the tasks presented in the framework and FSM repair via various methods. The paper presents experimental results with simulated data. These results and methods bring a new analysis and vision of LLMs that are useful for further development of machine learning technology and its applications to MDE.