NLLog: Lightweight, Explainable SOC Anomaly Detection via Log-to-Language Rewriting

2026-06-03

AI Summary

This study addresses the challenge in Security Operations Centers (SOCs) where system logs, often structured in rigid templated formats, hinder both automated analysis and human interpretability, thereby limiting anomaly detection efficiency. To bridge this gap, the authors propose an auditable, deterministic log rewriting mechanism that transforms parsed log templates into natural-language-style WHO-WHAT-SEVERITY sentences. This approach integrates lightweight dense encoding with coverage validation to yield an interpretable and pre-deployment-verifiable log representation layer. Building upon this foundation, the method employs TF-IDF weighting, tree-based ensemble classification, and TreeSHAP for post-hoc explainability, achieving low-latency, low false-positive anomaly detection. Experimental results demonstrate superior performance over existing baselines on the HDFS, BGL, and AIT datasets, highlighting its suitability for SOC triage scenarios.

Abstract

System-generated logs underpin security monitoring, yet their rigid template-based format hinders both automated analysis and human comprehension. We present NLLog (Natural-Language Log), a lightweight pipeline that deterministically rewrites parsed templates into WHO-WHAT-SEVERITY sentences, pools them with term-frequency-inverse-document-frequency weighting, classifies sessions with tree ensembles, and back-projects evidence with TreeSHAP for analyst review. On Hadoop Distributed File System (HDFS) and Blue Gene/L (BGL) corpora, NLLog exceeds two reproduced matched-protocol baselines; across HDFS, BGL, and the AIT Alert Data Set, it sustains low false-positive rates with commodity-hardware latency suitable for security operations center triage. Coverage, sparse-versus-dense, faithfulness, and adversarial ablations show that fallback sufficiency is corpus-dependent, that an enrollment-time coverage check can surface refinement requirements before deployment, and that an auditable deterministic rewrite combined with lightweight dense encoding provides a measurable representation layer for log-anomaly detection and triage.

Problem

• log anomaly detection
• security operations center
• log interpretability
• system logs
• false-positive rate

Innovation

• log-to-language rewriting
• explainable anomaly detection
• deterministic log transformation
• TreeSHAP interpretability
• lightweight SOC triage