checklist creation

In software development, selecting appropriate security features is challenging due to ambiguous standards, framework-specific vulnerabilities, and the absence of traceable, implementation-level security feature identifiers. Method: This paper proposes a fine-grained, implementation-oriented security feature taxonomy comprising 68 extensible characteristics; establishes systematic mappings between these features and major security standards (e.g., ISO/IEC 27001, NIST SP 800-53); and conducts reverse-engineering–driven, source-code–based feature modeling to empirically assess the support coverage of 21 widely adopted frameworks (e.g., Spring Security, OAuth 2.0). Contribution/Results: We present the first three-layer alignment—across security standards, framework capabilities, and source-code–level features—enabling precise security feature selection, implementation-level traceability, and long-term compliance auditing. The resulting taxonomy provides a structured semantic foundation and end-to-end traceability for secure software engineering.

Early system design suffers from ambiguous access control requirements, uneven distribution of domain knowledge, and cognitive asymmetry among stakeholders. Method: This paper proposes a tool-supported method that embeds asset and goal models as boundary objects into the requirements and architecture phases. It introduces a lightweight UML/SysML-inspired modeling language to explicitly represent access control concerns and integrates model comparison and difference analysis to enhance knowledge transparency and stakeholder alignment. Contribution/Results: The approach enables “shifting access control left” in the development lifecycle. Evaluated in a military aviation reusable component adaptation case, it identified six categories of latent access constraint conflicts, reduced requirement clarification cycles by 40%, and significantly improved cross-role collaboration efficiency.

This work addresses the absence of a unified, composable framework for generating and evaluating checklists tailored to fine-grained model assessment, which has hindered their application in scenarios such as model alignment and reinforcement learning. The authors propose a modular pipeline architecture—comprising a generator, a refiner, and a scorer—that leverages five abstract prompt templates to enable flexible configuration. This design unifies diverse generation strategies for the first time and supports cross-domain adaptation. The system integrates the LLM-as-a-Judge paradigm and is compatible with multiple backends, including OpenAI, OpenRouter, and vLLM, offering access via API, CLI, and a web interface. Experimental results demonstrate that the generated checklists exhibit strong alignment with human preferences and effectively validate their utility and generalization in real-world settings, such as ICLR peer-review rebuttals.

This study addresses the limitations of existing threat hunting tools in supporting cognitive externalization, clue organization, and cross-session continuity, which hinder effective collaboration and reasoning among threat hunters. Building upon prior research on mental model construction and sharing, the authors propose six design heuristics to evaluate the cognitive support capabilities of threat hunting tools. Guided by these heuristics, they developed and implemented a prototype system, Threat Hunter Board. Employing the design science research paradigm and cognitive walkthroughs, the development and preliminary evaluation demonstrate the tool’s feasibility in enhancing cognitive and collaborative workflows. This work lays the groundwork for future empirical studies with professional users.

This work addresses the limitations of existing evaluation methods, which focus narrowly on task completion and fail to ensure trustworthy deployment of embodied agents in multi-step, externally impactful scenarios, while also lacking coordination among evaluation, governance, orchestration, and runtime assurance. To bridge this gap, the paper proposes an integrated four-layer framework that establishes, for the first time, a closed-loop mechanism linking governance obligations to verifiable execution. Guided by the ODTA principles—Observability, Decidability, Timeliness, and Attestability—the framework introduces runtime localization testing and minimal action evidence bundles. Through a human-in-the-loop evidence synthesis approach, it formally connects policy requirements to concrete agent behaviors, exposing critical gaps such as the inability of static permissions and prompts to govern path-dependent actions. Validation via an enterprise procurement agent demonstrates the framework’s capacity to unify safety, robustness, and trajectory-level evaluation.

Existing cybersecurity response playbooks are predominantly heterogeneous and non-machine-readable, severely impeding automation and interoperability in SOAR platforms. To address this, we propose a modular, LLM-driven translation framework leveraging prompt engineering to automatically convert unstructured playbooks into standardized CACAO format. Our method innovatively integrates syntactic validation, semantic fidelity constraints, and multi-turn iterative refinement to ensure accurate structural reconstruction of complex control flows and preservation of operational semantics. Evaluated on a curated benchmark dataset, our approach significantly outperforms baseline models: it reduces CACAO syntax error rate by 72%, improves recall of critical procedural nodes by 39%, and supports end-to-end deployment. This work establishes a reusable technical pathway for structured representation and automated orchestration of security knowledge, advancing the operationalization of human-authored playbooks in autonomous security systems.

This work addresses the challenge of inconsistent README quality, which stems from varying audiences and usage contexts, and the inability of existing tools to simultaneously accommodate style, content, and contextual appropriateness. The paper proposes LintMe, a novel linter that uniquely integrates programmatic rules with large language model (LLM)-based content understanding. LintMe enables users to define context-sensitive checking rules via a lightweight domain-specific language (DSL), combining programmatic validations—such as link verification—with LLM-driven semantic assessments like terminology recognition. This approach enhances documentation quality while preserving authorial autonomy. A user study (N=11) demonstrates that LintMe is both usable and flexible, significantly outperforming baseline approaches that rely solely on direct LLM usage, and its scalability is further validated through illustrative case studies.

This study addresses the challenges users face in understanding the scope of permissions, real-time actions, and persistent post-task effects of personalized AI agents, which lead to inadequate risk awareness and difficulties in auditing. Through analysis of multi-source corpora and user interviews, the work identifies critical gaps in users’ mental models regarding agent capabilities, autonomy, permissions, and persistence. To bridge these gaps, the authors propose AgentTrace—a novel end-user visualization framework that integrates risk awareness with traceability, transparently surfacing permission histories, resource access patterns, and lasting side effects. Prototype evaluations demonstrate that AgentTrace significantly enhances users’ comprehension of agent behavior, effectively supports anomaly detection, and fosters calibrated trust in autonomous systems.

This work addresses the "privilege laundering" problem in tool-augmented agents, where individually benign tool permissions can combine to enable unsafe behaviors. The authors propose a transparent runtime defense mechanism that requires no modifications to either the agent or tool servers. Built upon an MCP proxy, the approach introduces receiver-oriented capability budgets and enforces monotonic capability decay through intersection-based propagation, ensuring that combined permissions can only weaken—not strengthen—during composition. By integrating explicit information flow control with a trusted allowlist, the method reduces attack success rates from 25–68% to 0–4.8% across 82 tasks while maintaining normal task completion rates of 96–100%, substantially outperforming existing baselines.

This work addresses the limitations of existing large language models, which are typically confined to isolated tasks and struggle to integrate into industrial-scale, multi-stage security workflows. To bridge this gap, the authors propose the first role-based multi-agent framework tailored to the entire vulnerability lifecycle, incorporating specialized agents—Planner, Analyzer, Fixer, and Verifier—augmented with CodeQL static analysis for enhanced precision. By introducing a role-oriented multi-agent architecture into end-to-end vulnerability management, this approach effectively aligns the capabilities of large models with real-world security engineering demands. Evaluated on 25 real-world C/C++ vulnerabilities, the system achieves a detection accuracy of 44%—comparable to GPT-5.5—and a repair accuracy of 19%, offering a practical and collaborative paradigm for intelligent security operations.

This study addresses the trust and reliability challenges—such as hallucinations, output instability, and misalignment with existing workflows—that hinder the adoption of large language models (LLMs) in Security Operations Centers (SOCs). Through a six-month ethnographic field study embedded within a multinational enterprise SOC, the research identifies core pain points including repetitive tasks, data fragmentation, and tooling bottlenecks. Guided by Nonaka’s SECI model, the authors develop a sociotechnical co-creation framework that deeply integrates frontline practitioners into the design and iterative refinement of LLM-augmented tools. This approach significantly enhances tool interpretability and workflow alignment, reduces operational friction, and fosters sustained LLM adoption in real-world SOC environments, demonstrating that practitioner-centered co-creation can overcome critical barriers to deploying AI in high-reliability security contexts.