🤖 AI Summary
This work addresses the limited reliability of structured security artifacts—such as KQL queries and MITRE ATT&CK mappings—generated by large language models, which often fall short of production-grade requirements. To bridge this gap, the authors propose a lightweight verification framework that shifts the focus of quality assurance from generation to validation. The core innovations include a hybrid testing strategy integrating test-driven generation, deterministic program verification, and semantic evaluation by large language models, alongside an interpretable judging mechanism distilled from expert decision distributions. Deployed in Microsoft Sentinel’s production environment, the framework significantly enhances the reliability of three critical types of security artifacts, establishing professional-grade and scalable validation standards.
📝 Abstract
Generating structured artifacts with Large Language Models - e.g. database queries, threat framework mappings, entity schemas - is relatively straightforward; however, making them reliable enough for production deployments presents challenges. We present a lightweight framework based on a core principle: LLMs generate, we validate. This reframing shifts responsibility from generation quality to validation rigor. The framework rests on three key attributes: First, test driven generation: when tests fail, the LLM receives indicative error messages that expose why the output failed, enabling the LLM to understand its mistakes and refine subsequent attempts. Second, deterministic and LLM-based tests: deterministic tests catch heuristics that can be programmatically verified (schema, syntax, cross-reference), while LLM-based tests evaluate nuanced semantic and delicate features that resist programmatic inspection (intent alignment, logical consistency, domain correctness). Third, expert-distilled judges: LLM-based tests are calibrated to distill and replicate human expert decision distribution, transforming manual human quality gates into scalable, reusable evaluation proxies that reflect professional-grade validation standards. We demonstrate the framework on three artifact types in the security domain - KQL query generation, MITRE ATT\&CK mapping, and entity mapping - deployed in production at Microsoft Sentinel. We believe this framework can be applied beyond security to other artifact generation tasks, providing a path to reliable, high-quality outputs without sacrificing the efficiency gains of LLM generation.