Cybersecurity Detection Classification with Reasoning-enabled Language Models

2026-07-30

AI Summary

This work addresses alert fatigue and detection backlogs in Security Operations Centers by proposing an automated triage method that integrates chain-of-thought reasoning into large language models. It pioneers the application of chain-of-thought prompting to Windows endpoint threat detection, enhanced through automatic prompt optimization, self-training, and reinforcement learning fine-tuned with verifiable rewards. To mitigate confidence degradation during reasoning, an independent probability calibrator is introduced. Experimental results demonstrate that the proposed approach, under task-specific fine-tuning, substantially outperforms general-purpose large language models, achieving an overall accuracy of 82.6%. In high-confidence automated triage scenarios, it improves recall rates by 43.0% for benign samples and 18.3% for malicious ones.

Abstract

A major issue in Security Operations Centers (SOCs) is alert fatigue, as the number of detections reported is more than staff can triage in a given day. Prior work prompts or fine-tunes large language models (LLMs) to emit a triage label directly, but does not train them to reason about whether a detection is a genuine threat. We train a chain-of-thought (CoT) reasoning-enabled triage classifier on real, human-labeled Windows endpoint detections by combining automated prompt optimization, self-training, and reinforcement learning with verifiable rewards. We find that CoT reasoning also degrades the label-token probabilities that automated triage relies on, so we separately train a calibrator that reads the full reasoning trace and estimates the probability that the verdict is correct. Our system reaches 82.6% test accuracy and, at the high-confidence operating point that governs automated triage, improves benign recall by 43.0% and malicious recall by 18.3% over a direct-label LLM classifier. We further show that the trained calibrator is necessary - an untrained confidence judge collapses high-confidence recall to zero - and that a finetuned 30B model significantly outperforms frontier general-purpose models, motivating targeted training over scale.

Problem

• alert fatigue
• cybersecurity detection
• triage classification
• reasoning-enabled language models
• Security Operations Centers

Innovation

• Chain-of-Thought Reasoning
• LLM Calibration
• Cybersecurity Triage
• Reinforcement Learning with Verifiable Rewards
• Alert Fatigue Mitigation