runtime monitoring

Machine learning systems lack auditability concerning transparency, fairness, and accountability. Method: This paper introduces a novel approach that systematically embeds responsible AI metrics—such as bias, explainability, and decision provenance—into logging infrastructure. Unlike conventional operational logs, the proposed framework integrates software engineering logging practices with AI ethics assessment dimensions, yielding a structured log model enabling continuous monitoring, traceable verification, and dynamic compliance checking. Contribution/Results: It represents the first method to achieve deep synergy between AI governance metrics and logging infrastructure, bridging the audit gap between model behavior and ethical compliance. Empirical evaluation demonstrates significant improvements in verifiability during regulatory audits and stakeholder trust. The approach provides actionable, implementation-ready guidance for developers and toolchain designers to enhance algorithmic accountability.

This paper addresses the fundamental limitation in runtime monitoring that branching-time properties—such as those expressible in modal μ-calculus—are inherently unmonitorable over a single execution trace. To overcome this, we propose a novel multi-round execution monitoring paradigm. Integrating monitoring theory, formal semantics, and game theory, we establish—for the first time—a precise theoretical characterization linking the syntactic structure of branching-time formulas to the minimum number of execution rounds required for monitoring, and rigorously prove that multi-round monitoring strictly extends classical monitorability boundaries. Our main contributions are: (1) a systematic characterization of observational power in multi-round monitoring; (2) tight upper and lower bounds on the minimal round complexity; and (3) confirmation that several canonical branching-time properties—including key safety and liveness specifications—become effectively monitorable within two or three rounds. This work provides both a theoretical foundation and a practical methodology for dynamic verification of complex concurrent and interactive behaviors.

Detecting runtime control-flow anomalies in complex systems remains challenging due to “unknown unknowns”—unforeseen deviations beyond predefined specifications. Method: This paper proposes a software monitoring approach integrating large language models (LLMs) with conformance checking. The method leverages LLMs to automatically align design models with source code, generate semantically consistent instrumentation strategies, and construct interpretable, lightweight control-flow models from event logs. Contribution/Results: It is the first work to introduce LLM-driven design-code co-modeling into dynamic monitoring—replacing manual rule specification with end-to-end automated monitor synthesis. Evaluated on a railway traffic management case study, the approach achieves 84.78% control-flow coverage, 96.61% F1-score, and 93.52% AUC for anomaly detection, significantly enhancing system reliability and trustworthiness under unknown environmental conditions.

To address the challenge of root-cause localization for iCFTL state specification violations, this paper proposes a static-dynamic collaborative diagnosis method based on backward data-flow analysis. First, backward data-flow analysis statically identifies potential violation-inducing statements; then, program instrumentation and runtime validation generate enriched execution traces to enable precise attribution. This work is the first to apply backward data-flow analysis to iCFTL specification violation diagnosis, significantly improving both diagnostic accuracy and interpretability. Experimental evaluation across 10 real-world projects and 112 specifications demonstrates that the approach precisely localizes violating statements in 90% of cases, reduces manual code review effort by over 90%, achieves an average diagnosis time of ≤7 minutes, incurs memory overhead <25 MB, and imposes runtime performance overhead <30%.

This study addresses the challenge of effectively monitoring early-stage agent systems, where structural flaws often obscure task-level errors. The authors propose a three-dimensional (quality, suitability, efficiency) and three-granularity (intra-run, inter-run, structural) monitoring and triaging framework tailored for low-maturity agent systems. They introduce a novel system maturity staging model based on the coefficient of variation and monitoring granularity, integrated with a severity classification adapted from FMEA to guide human review. The resulting transferable monitoring architecture supports document-driven, multi-stage workflows, enhanced by a synthetic testbed with controlled error injection. Experimental results demonstrate that structural defects significantly mask task-level signals; 97% of issues can be automatically traced, with only 2% requiring human intervention, and each granularity level precisely identifies its corresponding defect type (coefficients of variation: 0.02, 1.25, and 0.00, respectively).

Existing runtime enforcement techniques struggle to handle reactive systems with complex continuous dynamics and lack effective mechanisms for intervening in hybrid behaviors. This work proposes the first framework that integrates hybrid automata into runtime enforcement, enabling coordinated discrete event editing and continuous-time monitoring to correct system behavior at any instant by suppressing, delaying, or inserting events. The paper establishes formal enforceability conditions and devises an online strategy synthesis algorithm based on reachability analysis. Evaluation on an adaptive cruise control case study demonstrates that the approach ensures safety properties even when the underlying controller is unsafe, all while incurring minimal computational overhead.

This work addresses the pervasive issue of redundant and isolated messages in system logs, which hinder downstream tasks such as model reasoning and anomaly detection. To tackle this challenge, the authors propose LogPurifier—the first task-agnostic log cleansing framework—that systematically purifies logs by extracting log templates and modeling their dependencies to accurately identify and remove messages irrelevant to system functional behavior. By doing so, LogPurifier enables effective log sanitization applicable across diverse analytical scenarios. Experimental results demonstrate that LogPurifier substantially improves both accuracy and efficiency in various downstream tasks, thereby validating its effectiveness and generalizability.

This work addresses the challenge of enforcing temporal safety constraints throughout the lifecycle of black-box AI systems, such as large language models (LLMs), which are inherently difficult to verify. The paper proposes the first offline auditing and online monitoring framework that integrates Linear Temporal Logic (LTL) with machine learning. This framework enables formal verification of complex temporal behavioral specifications by introducing a sampling-driven predictive monitor and an intervenable runtime monitor, effectively preventing policy violations. Experimental results demonstrate that the proposed approach significantly outperforms existing LLM-based evaluators in detecting temporal violations. Notably, it achieves performance on par with or superior to state-of-the-art large models using only a small labeled model, while its intervention mechanism substantially reduces violation rates without compromising task performance.

This work proposes an automated log aggregation and analysis framework based on large language models to address the growing challenge of log analysis in increasingly complex systems, where engineers traditionally rely on domain expertise to manually craft intricate LogQL queries. The framework enables end-to-end generation of LogQL queries from natural language instructions by integrating a hierarchical log knowledge base, natural language understanding, knowledge retrieval, and tool invocation mechanisms. Evaluated on four real-world log datasets, the approach achieves an average accuracy of 76.8%, significantly outperforming existing baselines and demonstrating its effectiveness and practicality for log analysis tasks.

Automotive electronic control units (ECUs) are intricate systems with hundreds of individual functions, numerous software components, and multiple interdependent tasks. A prevalent structural pattern in these systems are so-called cause-effect chains. While significant research efforts have been dedicated to the temporal analysis and optimization of these chains, particularly minimizing data age and function response times, other crucial non-functional properties remain relatively underexplored. In particular, the safety integrity level (SIL) classification substantially influences the system design by determining task colocation strategies. Improper sharing of functions or interweaving tasks with different safety levels can compromise the integrity of critical functions. Additionally, AUTOSAR basic software (BSW) (e.g. OS, runtime environment, communication stacks, or diagnostics) introduces complexity that varies based on task characteristics and SIL categories. Furthermore, memory requirements present another critical challenge, given the diversity of memory architectures and SIL-specific dependencies that strongly constrain task allocations. This paper thoroughly characterizes a real-world automotive application, describing an automotive application based on SIL constraints, the impact of basic software, and memory requirements. In this context, the Driverator configuration framework is introduced for scalable system analysis.