What is Formal Verification without Specifications? A Survey on mining LTL Specifications

2025-01-27

AI Summary

This study addresses the high cost, error-proneness, and poor maintainability of manually writing Linear Temporal Logic (LTL) specifications—a core bottleneck in formal verification. We systematically survey and evaluate automated LTL specification mining methods. First, we propose a unified, multi-paradigm classification framework—covering constraint solving, neural networks, enumeration-based search, formal language inference, and grammar-guided learning—marking the first such comprehensive taxonomy. Second, we introduce a standardized evaluation methodology assessing scalability, interpretability, and noise robustness across approaches. Our analysis clarifies the applicability boundaries and inherent limitations of each paradigm and yields a practical, industry-oriented selection guide. The work significantly advances the automation level and reliability of LTL specification acquisition, bridging the gap between theoretical mining techniques and real-world verification practice.

Technology Category

• Knowledge Representation and Reasoning: Automated Reasoning and Theorem Proving
• Constraint Satisfaction and Optimization: Satisfiability Modulo Theories
• Data Mining & Knowledge Management: Mining of Spatial, Temporal or Spatio-Temporal Data

Application Category

• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Virtually all verification techniques using formal methods rely on the availability of a formal specification, which describes the design requirements precisely. However, formulating specifications remains a manual task that is notoriously challenging and error-prone. To address this bottleneck in formal verification, recent research has thus focussed on automatically generating specifications for formal verification from examples of (desired and undesired) system behavior. In this survey, we list and compare recent advances in mining specifications in Linear Temporal Logic (LTL), the de facto standard specification language for reactive systems. Several approaches have been designed for learning LTL formulas, which address different aspects and settings of specification design. Moreover, the approaches rely on a diverse range of techniques such as constraint solving, neural network training, enumerative search, etc. We survey the current state-of-the-art techniques and compare them for the convenience of the formal methods practitioners.

Problem

• Automatic Generation
• Temporal Logic Language (LTL)
• System Behavior Analysis

Innovation

• Automated Learning
• Temporal Logic Languages (LTL)
• Design Verification