diagnostics

Designs and implements tests, monitoring tools, and procedures to detect, measure, and localize faults or anomalous behavior in software, models, or systems; analyzes logs, metrics, traces, and diagnostic outputs to determine root causes, quantify impacts, and produce actionable remediation or mitigation steps.

diagnostics

Recent Skill Trend

Momentum and market value over time
Trending
Score
No comparison yet
1.34
Oct 01, 2026Oct 01, 2026
Career
Value
No comparison yet
$203K/year
Oct 01, 2026Oct 01, 2026

Must-Read Papers

Most classic and influential ideas
View more

This work addresses the pervasive issue of redundant and isolated messages in system logs, which hinder downstream tasks such as model reasoning and anomaly detection. To tackle this challenge, the authors propose LogPurifier—the first task-agnostic log cleansing framework—that systematically purifies logs by extracting log templates and modeling their dependencies to accurately identify and remove messages irrelevant to system functional behavior. By doing so, LogPurifier enables effective log sanitization applicable across diverse analytical scenarios. Experimental results demonstrate that LogPurifier substantially improves both accuracy and efficiency in various downstream tasks, thereby validating its effectiveness and generalizability.

downstream tasksirrelevant messageslog analysis

How Execution Features Relate to Failures: An Empirical Study and Diagnosis Approach

Feb 25, 2025
MS
Marius Smytzek
🏛️ CISPA Helmholtz Center for Information Security | Humboldt-Universität zu Berlin

This paper addresses the dual challenges of low fault localization accuracy and weak root-cause interpretability in software debugging. To this end, we propose an interpretable diagnosis method based on multi-execution feature fusion. Through empirical analysis of 310 real-world defects, we first establish—systematically and for the first time—that scalar pairs constitute the strongest failure-correlated features. Building upon this insight, we design a joint modeling framework that integrates 17 fine-grained execution features, including variable values, branch conditions, and definition-use chains. We further develop a feature-importance-driven interpretable decision tree model that automatically generates human-readable diagnostic rules. Evaluation across 20 open-source projects demonstrates that our approach significantly improves both fault localization accuracy and root-cause identification depth, substantially reducing developer debugging time. The method achieves a favorable balance between high precision and strong interpretability.

Analyzing diverse execution featuresDeveloping interpretable debugging diagnosesEnhancing fault localization accuracy

This study addresses the challenge of effectively monitoring early-stage agent systems, where structural flaws often obscure task-level errors. The authors propose a three-dimensional (quality, suitability, efficiency) and three-granularity (intra-run, inter-run, structural) monitoring and triaging framework tailored for low-maturity agent systems. They introduce a novel system maturity staging model based on the coefficient of variation and monitoring granularity, integrated with a severity classification adapted from FMEA to guide human review. The resulting transferable monitoring architecture supports document-driven, multi-stage workflows, enhanced by a synthetic testbed with controlled error injection. Experimental results demonstrate that structural defects significantly mask task-level signals; 97% of issues can be automatically traced, with only 2% requiring human intervention, and each granularity level precisely identifies its corresponding defect type (coefficients of variation: 0.02, 1.25, and 0.00, respectively).

Agentic SystemsMonitoringStructural Defects

LogLM: From Task-based to Instruction-based Automated Log Analysis

Oct 12, 2024
YL
Yilun Liu
🏛️ Huawei | Nankai University

Existing log analysis models are task-specific, rely heavily on domain-specific annotated data, exhibit poor generalization, and struggle with complex or unseen instructions. Method: We propose LogLM, an instruction-driven large language model for log analysis, which unifies diverse log tasks—including anomaly detection, parsing, and summarization—into a standardized instruction-response format. LogLM is adapted to the log domain via multi-task instruction tuning and log-specific instruction engineering. It accepts natural-language instructions and supports zero-shot cross-task transfer. Contribution/Results: Experiments demonstrate that LogLM outperforms all state-of-the-art methods across five core log analysis tasks. It exhibits strong generalization to complex instructions and previously unseen tasks. As a single unified model, LogLM replaces multiple specialized models, significantly improving deployment efficiency and task-agnostic capability.

Log AnalysisModel AdaptabilityTask Generalization

Automated Defects Detection and Fix in Logging Statement

Aug 06, 2024
RZ
Renyi Zhong
🏛️ The Chinese University of Hong Kong | Singapore Management University

Low-quality log statements—such as ambiguous or misleading ones—obscure actual program behavior and impede software maintenance. Prior work primarily focuses on detecting single log defects and relies on manual fixes. This paper proposes LogFixer, the first automated two-stage framework targeting four real-world log defects: detection and repair. In the offline stage, a lightweight similarity classifier is trained on synthetically defective logs; in the online stage, problematic logs are identified via joint modeling of static textual features and dynamic variable contexts, and semantically appropriate repairs are recommended using large language models (LLMs). LogFixer innovatively integrates a lightweight classifier with LLMs in a synergistic paradigm, ensuring robust detection while enhancing repair validity. Evaluation shows an F1-score of 0.625; adoption rates of static and dynamic repair suggestions improve by 48.12% and 24.90%, respectively; repair suggestion adoption reaches 61.49% on unseen projects; and 40 fixes submitted to GitHub have yielded 25 merged confirmations.

Detects and repairs defects in logging statements automaticallyIdentifies four types of logging defects via log-centric analysisImproves log quality using LLM-based recommendations and fixes

Latest Papers

What's happening recently
View more

Bug fixing is a complex and time-consuming task in software development. Bug localization research tends to focus on the accuracy of automated tools that suggest source code files for developers to look at. However, little is known about how developers use these tools in practice. This paper reports on an ongoing qualitative user study. Eleven participants worked through four realistic bug localization tasks in a controlled environment and were given varying levels of support information offered by a specialized tool. Participants were asked to think aloud in a semi-structured interview session. The preliminary findings provide insight into three aspects of practice: how developers interact with tools, the role social and contextual information plays, and problem solving. The study demonstrates that bug localization is complex and suggests that the adoption of effective tools depends on more than their accuracy.

bug localizationdeveloper behaviourqualitative study

Existing debugging tools excel at verifying hypotheses but struggle to support hypothesis generation, as programmers must manually reconstruct the program’s state evolution. This work proposes a novel debugging paradigm centered on complete execution traces, leveraging program tracing techniques to record and temporally visualize the actual code paths executed, rather than relying on the static structure of the source code. By presenting runtime behavior in a chronological and contextualized manner, this approach significantly enhances the comprehensibility of program execution, thereby facilitating more efficient hypothesis generation during debugging. We implement a prototype system and conduct preliminary experiments that demonstrate its effectiveness in improving program understanding efficiency, while also uncovering key challenges and promising directions for future research.

debuggingexecution tracehypothesis generation

This study addresses the limitations of traditional statistical fault localization (SFL), which relies solely on code execution traces and often fails to accurately pinpoint root causes. To overcome this, the authors systematically incorporate execution features—such as data flow, variable values, and branch conditions—extracted via the EFDD tool from the Tests4Py dataset. They train project-specific random forest models and map feature importance back to source code lines, integrating these insights with classical SFL formulas to enhance localization accuracy. Rigorous evaluation is conducted using a confounder-adjusted mixed-effects model and paired statistical tests. Experimental results demonstrate that the proposed approach significantly improves the accuracy of reference patches while reducing inspection effort at both line and function levels, confirming its robustness and practicality across multiple dimensions.

Developer Inspection EffortExecution FeaturesFault Localization Accuracy