design audit protocols

This work addresses the critical yet often overlooked security risks in large language model (LLM) agent systems, which frequently stem from software stack components such as tool code, deployment configurations, and permission settings—not merely from the underlying models. To this end, we present the first dedicated security analysis framework tailored for LLM-based agent applications. Our system integrates dataflow analysis, credential detection, structured configuration parsing, and permission risk assessment to precisely identify diverse vulnerabilities across tool functions, prompts, and deployment artifacts, outputting results in the standardized SARIF format. Evaluated on 22 real-world samples containing 42 annotated vulnerabilities, our approach successfully detects 40 true positives with only 6 false positives, substantially outperforming general-purpose static application security testing (SAST) tools while completing each scan in under one second.

This study addresses the critical lack of accountability in large language model (LLM) agents following external actions, which hinders traceability and responsibility attribution. The work introduces the first systematic framework for agent auditability, articulating five core dimensions and proposing an “Auditability Card” to standardize assessment. It further develops a full-lifecycle auditing architecture integrating detection, enforcement, and recovery mechanisms. Leveraging runtime intervention, tamper-proof logging, and log-recovery techniques—validated through ecosystem-wide security evaluations and controlled experiments—the study identifies 617 security flaws across mainstream open-source LLM agent projects. Experimental results demonstrate that a pre-execution mediation layer incurs only 8.3 milliseconds of overhead and that partial reconstruction of accountability-critical information remains feasible even in the absence of complete logs.

This work addresses the lack of traceable and tamper-resistant transparency mechanisms in large language models (LLMs) deployed in high-stakes decision-making contexts, which undermines accountability. To bridge this gap, the paper introduces the first LLM lifecycle auditing framework that integrates technical provenance with governance records. It proposes a reference architecture enabling cross-organizational traceability and implements a lightweight, open-source Python-based auditing layer. By leveraging append-only logs, event emitters, structured metadata, and an auditor interface, the system seamlessly integrates into existing LLM workflows with minimal intrusiveness. This design ensures complete, tamper-evident traceability across critical stages—including training, deployment, and monitoring—thereby facilitating robust accountability and responsibility attribution throughout the model’s lifecycle.

This work addresses the structural security vulnerabilities in AI-generated code, which, despite functional correctness, often lacks robust safety guarantees. Existing LLM-based security review methods suffer from inconsistent coverage, irreproducibility, and a lack of verifiable evidence chains or audit trails. To overcome these limitations, the paper proposes a four-stage security auditing pipeline grounded in event sourcing. By leveraging event-driven mechanisms, immutable logging, and protocol-enforced constraints, the approach transforms informal, conversational code reviews into a structured, verifiable, and replayable governance process. The system integrates constrained agent outputs, append-only logs, hash-based verification, and replay consistency checks, covering 16 security domains and 95 executable checks. It produces comprehensive audit artifacts—including vulnerability inventories, risk matrices, remediation recommendations, and machine-readable Markdown/JSON reports—enabling full traceability and reproducibility throughout the auditing lifecycle.

This work addresses the challenge that ambiguous requirements in natural language specifications often lead to consistent errors across multiple implementations, which traditional differential testing fails to detect. To this end, the authors propose SPECA, a novel framework that automatically translates informal specifications into structured checklists and maps them to critical code locations across diverse implementations, enabling checklist-driven, one-to-many cross-implementation auditing. SPECA integrates natural language processing, threat modeling, and agent-based automated auditing to construct an end-to-end specification alignment verification system. Evaluated on the Ethereum Fusaka upgrade, the approach identified 76.5% of valid vulnerabilities through cross-implementation checks, and its optimized auditing agent achieved a 27.3% recall rate on high-severity vulnerabilities, outperforming 96% of human auditors.

This study addresses the challenge in IT auditing where evidence from heterogeneous organizations is fragmented and compliance with security and regulatory controls must be assessed based on semantic adequacy rather than keyword matching, hindering automation. To tackle this, the work proposes the first system integrating Retrieval-Augmented Generation (RAG) with a multi-agent collaboration framework. The system orchestrates evidence retrieval, evaluation generation, adversarial challenge of assertions, and resolution of disagreements to produce interpretable audit recommendations that include citations, reasoning, gap analysis, and remediation guidance. Experimental validation under the ISO/IEC 27001 standard demonstrates that the approach effectively supports control interpretation and audit preparation, significantly improving efficiency. Nevertheless, human oversight remains necessary to calibrate judgments of evidentiary sufficiency.

This study addresses the high cost and limited scalability of manual compliance audits under Germany’s IT-Grundschutz framework, which pose a significant burden on small and medium-sized enterprises. To partially automate the certification process—encompassing structural analysis, protection requirements assessment, modeling, and compliance verification—the authors propose a multi-agent system (MAS) integrated with a hybrid retrieval-augmented generation (HybridRAG) approach. The method innovatively incorporates a hypothesis-validation loop to mitigate agent hallucinations and employs a decoupled reasoning pipeline that separates semantic extraction from deterministic inheritance of protection requirements, thereby enhancing compliance rigor. Experimental results demonstrate a substantial reduction in human effort for semantic tasks; however, stages relying on deterministic logic remain constrained by the inherent probabilistic nature of large language models.