Problem
Research questions and friction points this paper is trying to address.
Log Anomaly Detection
Large Language Models
Robustness
Efficiency
Empirical Study
Towards Understanding LLM-Based Log Anomaly Detection: An Empirical Study of Performance, Efficiency, and Robustness
🤖 AI Summary
This study addresses the unclear impact of adaptation strategies, architectures, and deployment configurations on large language model (LLM)-based log anomaly detection through a systematic empirical investigation. Leveraging multiple datasets alongside parameter scaling analysis and noise perturbation testing, it comprehensively evaluates how diverse adaptation strategies, model scales, and quantization settings jointly affect detection accuracy, efficiency, and robustness. The findings reveal substantial computational cost disparities among models achieving comparable accuracy and demonstrate that low-bit quantization effectively preserves detection performance. By elucidating critical trade-offs across various configurations, this work provides essential empirical evidence and practical guidance for developing efficient and robust LLM-based log anomaly detection systems.
📝 Abstract
Large language models (LLMs) have demonstrated promising performance in log anomaly detection, yet how their adaptation strategies, architectures, and deployment configurations affect detection effectiveness remains insufficiently understood. To investigate these factors, we conduct a systematic empirical analysis across three public log datasets, examining different adaptation strategies, model architectures, parameter scales, and quantization settings. Our results reveal substantial performance differences across adaptation strategies, while model scaling yields varying detection gains across datasets. We further observe that models with comparable detection accuracy can exhibit markedly different computational costs, and that low-bit quantization largely preserves detection performance in the evaluated configurations. Finally, we examine detection robustness under structural, semantic, and label noise at different perturbation levels. These findings provide empirical insights into the performance, efficiency, and robustness of LLM-based log anomaly detection, highlighting practical considerations beyond conventional accuracy-oriented evaluation.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Log Anomaly Detection
Large Language Models
Empirical Study
Quantization
Robustness
🔎 Similar Papers
No similar papers found.
