🤖 AI Summary
This work addresses inconsistencies arising from structural mismatches between natural language and formal languages during requirements formalization. It proposes a “consistency through formalization” principle, mandating strict logical alignment among natural language, the structured language FRETish, and the formal temporal logic MTL. Guided by this principle, the authors refine the FRETish-to-MTL translation pipeline in NASA’s FRET tool. Their approach uniquely integrates cross-layer consistency constraints into a collaborative framework combining large language models and formal verification tools. This integration not only uncovers and corrects multiple inconsistencies in the original translation but also demonstrates superior correctness and reliability, as substantiated by formal equivalence proofs and empirical evaluation.
📝 Abstract
Formalisation is the process of writing system requirements in a formal language. These requirements mostly originate in Natural Language. In the field of Formal Methods, formalisation is often identified as one of the most delicate and complicated steps in the verification process. Not seldomly, formalisation tools and environments choose various levels of requirement descriptions: Natural Language, Technical Language, Diagram Representations and Formal Language, to mention a few. In the literature, there are various maxims and principles of good practice to guide the process of requirement formalisation. In this paper we propose a new guideline: Coherency through Formalisations. The guideline states that the different levels of formalisation mentioned above should roughly follow the same logical structure. The principle seems particularly relevant in the setting where LLMs are prompted to perform reasoning tasks that can be checked by formal tools using Structured Natural Language to act as an intermediate layer bridging both paradigms. In the light of coherency, we analyze NASA's Formal Requirement Elicitation Tool FRET and propose an alternative automated translation of the Controlled Natural Language FRETish to the formal language of MTL. We compare our translation to the original translation and prove equivalence using model checking. Some statistics are performed which seem to favor the new translation. As expected, the translation process yielded interesting reflections and revealed inconsistencies which we present and discuss.