GRIDAI: Generating and Repairing Intrusion Detection Rules via Collaboration among Multiple LLM-based Agents

2025-10-15

AI Summary

Existing rule-based web intrusion detection systems suffer from rule redundancy and insufficient variant coverage due to neglecting semantic relationships between novel attacks and existing rules. To address this, we propose the first end-to-end large language model (LLM)-driven framework for automated rule generation and repair. Our approach employs a multi-agent collaboration mechanism integrating attack-sample relationship identification, tool-augmented syntactic/semantic validation, incremental signature updating, and real-time feedback verification—enabling adaptive evolution of rule sets. Innovatively, multiple specialized LLMs coordinate tasks—including attack-type classification, rule generation, and consistency checking—to mitigate hallucination while ensuring accuracy and interpretability. Experiments on public and private datasets demonstrate that our method reduces rule redundancy by 38.7% on average, improves variant attack detection by 22.4%, achieves 91.3% accuracy in rule generation and repair, and exhibits superior generalization over baseline approaches.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Generation
• Data Mining & Knowledge Management: Rule Mining & Pattern Mining

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Rule-based network intrusion detection systems play a crucial role in the real-time detection of Web attacks. However, most existing works primarily focus on automatically generating detection rules for new attacks, often overlooking the relationships between new attacks and existing rules, which leads to significant redundancy within the ever-expanding ruleset. To address this issue, we propose GRIDAI, a novel end-to-end framework for the automated Generation and Repair of Intrusion Detection rules through collaboration among multiple LLM-based agents. Unlike traditional methods, GRIDAI first assesses the nature of incoming attack samples. If the sample represents a new attack type, it is used to generate a new rule. Otherwise, the sample is identified as a variant of an attack already covered by an existing rule and used to repair the rule by updating the corresponding signature, thereby enhancing its generalization capability. Additionally, to mitigate syntactic and semantic errors in rules caused by LLM hallucinations, we incorporate a tool-based real-time validation mechanism and a representative attack sample maintained for each rule, enabling fully automated rule generation and repair. Comprehensive experiments were conducted on a public dataset containing seven types of attacks and a private dataset with 43 attack types. The results demonstrate that GRIDAI accurately identifies the relationships between new attack samples and existing rules, efficiently generates and repairs rules to handle new attacks and variants, and effectively mitigates the impact of LLM hallucinations.

Problem

• Automatically generating and repairing intrusion detection rules for network security
• Reducing rule redundancy by identifying relationships between new and existing attacks
• Mitigating LLM-induced syntactic and semantic errors in detection rules

Innovation

• Multi-agent LLM collaboration for rule generation
• Tool-based validation to reduce LLM hallucinations
• Automated rule repair for existing attack variants