design monitors

Design monitors: design and implement monitoring components that instrument internal states and output signals, specifying which signals are observable and how they are measured. Specify detection thresholds and decision rules, evaluate monitor reliability and performance, and integrate monitors into processing pipelines.

designmonitors

Recent Skill Trend

Momentum and market value over time
Trending
Score
No comparison yet
-0.1
Oct 01, 2026Oct 01, 2026
Career
Value
No comparison yet
$200K/year
Oct 01, 2026Oct 01, 2026

Must-Read Papers

Most classic and influential ideas
View more

This paper addresses the fundamental limitation in runtime monitoring that branching-time properties—such as those expressible in modal μ-calculus—are inherently unmonitorable over a single execution trace. To overcome this, we propose a novel multi-round execution monitoring paradigm. Integrating monitoring theory, formal semantics, and game theory, we establish—for the first time—a precise theoretical characterization linking the syntactic structure of branching-time formulas to the minimum number of execution rounds required for monitoring, and rigorously prove that multi-round monitoring strictly extends classical monitorability boundaries. Our main contributions are: (1) a systematic characterization of observational power in multi-round monitoring; (2) tight upper and lower bounds on the minimal round complexity; and (3) confirmation that several canonical branching-time properties—including key safety and liveness specifications—become effectively monitorable within two or three rounds. This work provides both a theoretical foundation and a practical methodology for dynamic verification of complex concurrent and interactive behaviors.

Extends monitorability of branching-time properties via multiple executionsInvestigates enhanced monitoring capabilities over multiple system runsLinks property syntax structure to required number of system runs

Quantitative and Approximate Monitoring

May 18, 2021
TA
Thomas A. Henzinger
🏛️ IST Austria

Existing runtime monitors support only Boolean specification verification, making it infeasible to progressively approximate quantitative properties—such as average response time—over infinite traces. Method: This paper establishes the first unified formal framework for quantitative approximate monitoring, introducing quantitative monitors whose estimates monotonically improve as observation prefixes grow, and rigorously modeling the trade-off between estimation accuracy and resource consumption (specifically, register count). Contribution/Results: We prove that register count strictly determines the theoretical upper bound on achievable accuracy; moreover, each additional register strictly increases the attainable precision—demonstrating an irreducible, non-compensatory relationship between resources and accuracy. Our framework conservatively extends classical Boolean monitoring theory while ensuring soundness. The proposed approach provides provably optimal, resource-bounded approximate monitoring for critical performance metrics, enabling verifiable, deployment-aware runtime assurance.

Analyzing precision-cost trade-offs in monitor resource usageDeveloping approximate monitors for numerical value estimationGeneralizing runtime verification to quantitative trace properties

This work addresses the challenges of high early-stage uncertainty in manufacturing monitoring system development—leading to redundant modeling and substantial training costs—and the limited transferability of filtering pipelines in cross-domain image segmentation tasks. To tackle these issues, the authors propose a problem-centric design paradigm that constructs an abstract system model to continuously accumulate and retrieve historical segmentation tasks along with their associated filtering pipelines, enabling solution reuse and incremental optimization. The approach integrates similarity-based problem retrieval, abstract modeling, pipeline reuse, and a retrieval-augmented evolutionary learning mechanism. Experimental results demonstrate that the method significantly reduces training costs and late-stage revision risks, provides the first systematic validation of filtering pipeline transferability across similar segmentation tasks, and achieves a favorable balance among complexity, technical requirements, and reliability under lightweight model constraints.

cross-domain transferabilityfilter pipelinesmonitoring systems

In software design, paradigm-implied semantic expectations—such as data abstraction consistency and feedback-control closed-loop behavior—are often left implicit, leading to design deviations and verification challenges. To address this, we introduce the concept of *design obligations*: explicit, logically formalizable, and verifiable specifications that codify such implicit constraints inherent to design paradigms. Leveraging formal modeling and paradigm semantics analysis, we establish two obligation frameworks—one for data-abstraction-based systems and another for feedback-driven adaptive systems—precisely capturing their core semantic requirements. We demonstrate that common design flaws stem from obligation violations and show how these obligations enable rigorous compliance verification and pedagogical application. This work bridges the semantic gap between design intent and implementation, providing both theoretical foundations and a methodological framework for paradigm-driven design assurance.

Addressing implicit or informal design expectations in software paradigms.Ensuring software designs meet semantic expectations beyond syntax.Introducing 'design obligations' to enforce proper paradigm use.

This work addresses the unreliability of Chain-of-Thought (CoT) monitors in detecting undesirable behaviors—such as test-time exploitation—often stemming from insufficient information extraction or poor approximation of the monitoring function. For the first time, it formalizes CoT monitorability from an information-theoretic perspective, establishing that non-zero mutual information between the CoT and the output is necessary but insufficient for effective monitoring. The study identifies two key error sources: information gaps and steering errors. To mitigate these, it proposes a novel label-free joint optimization framework that combines conditional mutual information maximization with oracle-guided reinforcement training to systematically enhance monitor performance. Experiments demonstrate that this approach significantly improves monitoring accuracy across diverse settings, effectively suppresses CoT degradation, and alleviates reward hacking even under imperfect reward signals.

Chain-of-Thoughtinformation theorymonitorability

Latest Papers

What's happening recently
View more

Large language models (LLMs) can still generate unsafe outputs in deployment, necessitating efficient real-time monitoring. This work proposes a lightweight online safety monitoring mechanism that integrates signals from an external verification model, threshold-based decision rules, and risk control theory to produce reliable alerts through calibrated thresholds. The approach features a simple architecture that avoids computationally intensive procedures yet achieves detection performance on par with state-of-the-art sequential hypothesis testing methods across mathematical reasoning and red-teaming benchmarks. By combining practical efficiency with theoretical guarantees, the proposed method offers a viable solution for real-world LLM safety monitoring.

Alarm SystemDeployment-time SafetyLarge Language Models

This work addresses the inefficiencies and semantic inconsistencies arising from separately implementing driver and monitor programs in traditional hardware module testing. To overcome this, the authors propose a domain-specific language (DSL) tailored to hardware communication protocols, which enables the unified specification of both driver and monitor logic through an imperative syntax, thereby ensuring their semantic consistency for the first time. Building upon this DSL, they develop a prototype tool that leverages waveform parsing and transaction-level trace inference techniques to accurately reconstruct protocol-compliant transaction sequences from raw signal waveforms. Experimental results demonstrate that the approach significantly improves development efficiency, with further validation planned on real-world interconnect protocols such as Wishbone and AXI-Stream.

driverhardware communicationmonitor

This work addresses the limitations of traditional structural coverage metrics in embedded software testing, which are often confined to the unit level and fail to reflect true coverage completeness in integration and system testing. Instrumentation-based approaches risk perturbing runtime behavior, while pure tracing techniques suffer from unreliability under high compiler optimization. To overcome these challenges, the paper proposes an integration-test-driven coverage strategy featuring a novel “integration-first” closed-loop workflow. By synergistically combining embedded tracing with hybrid runtime analysis (hRA) to preserve semantic boundaries, and leveraging source-to-target mapping for evidential traceability alongside Hyper Coverage for cross-variant merging, the approach establishes a unified evidence-integration mechanism. Evaluated on -O3-optimized release binaries, it reliably achieves branch, condition, and MC/DC coverage measurements and precisely identifies source code lines consistently uncovered across all variants, thereby significantly enhancing confidence in the test completeness of embedded systems.

compiler optimizationembedded softwareintegration testing

Hot Scholars

JE

Joshua Engels

Google Deepmind
Mechanistic InterpretabilityAI Safety
LX

Luyi Xing

Associate Professor of Computer Science, University of Illinois Urbana-Champaign/Indiana University
System SecurityData Privacy and Cybercrime
XQ

Xue Qin

Assistant Professor, Villanova University
Software EngineeringSoftware Testing
XL

Xiaojing Liao

University of Illinois Urbana-Champaign
Cybersecurity
NP

Nils Palumbo

PhD Student in Computer Science, UW-Madison