operationalize policies

Enterprise-scale general-purpose agents lack built-in, reusable governance mechanisms for autonomous cross-tool operation, making it difficult to satisfy requirements for compliance, auditability, and behavioral controllability. This work proposes the CUGA policy system, which embeds runtime governance capabilities into five critical checkpoints of the agent execution pipeline—intent protection, playbook guidance, tool invocation control, human approval gating, and output formatting—through a modular “policy-as-code” architecture. Without requiring model fine-tuning, CUGA enables proactive, continuous, and structured behavior control. By integrating typed governance primitives, dynamic playbook injection, and human-in-the-loop approval, the system effectively blocks malicious requests, enforces structured tool sequences, and triggers manual review for high-risk operations in healthcare scenarios, significantly enhancing policy adherence, execution consistency, and deployment safety.

Large language model (LLM) agents exhibit unreliable adherence to corporate policies in business process automation. To address this, we propose a deterministic, transparent, and modular policy compliance framework comprising two phases: (1) an offline phase that compiles natural-language policy documents into verifiable guard code, and (2) a runtime phase that inserts lightweight, policy-agnostic guards before tool invocation—thereby decoupling policy enforcement from agent logic. This design enhances interpretability, maintainability, and agility in policy updates. Experiments on the τ-bench Airlines testbed demonstrate the framework’s effectiveness in intercepting policy-violating actions, validating its feasibility. However, empirical evaluation also uncovers critical deployment challenges, including incompleteness in policy coverage and difficulties in dynamically adapting guards to contextual changes. The framework thus advances policy-aware LLM agent deployment while surfacing key open issues for future work.

This work addresses the limitations of existing evaluation methods, which focus narrowly on task completion and fail to ensure trustworthy deployment of embodied agents in multi-step, externally impactful scenarios, while also lacking coordination among evaluation, governance, orchestration, and runtime assurance. To bridge this gap, the paper proposes an integrated four-layer framework that establishes, for the first time, a closed-loop mechanism linking governance obligations to verifiable execution. Guided by the ODTA principles—Observability, Decidability, Timeliness, and Attestability—the framework introduces runtime localization testing and minimal action evidence bundles. Through a human-in-the-loop evidence synthesis approach, it formally connects policy requirements to concrete agent behaviors, exposing critical gaps such as the inability of static permissions and prompts to govern path-dependent actions. Validation via an enterprise procurement agent demonstrates the framework’s capacity to unify safety, robustness, and trajectory-level evaluation.

To address insufficient stakeholder engagement and the lack of iterative validation in large language model (LLM) alignment, this paper introduces the “policy prototyping” paradigm—a human-centered, collaborative framework for designing LLM behavioral policies. Methodologically, it integrates human-AI co-design, rapid policy sandbox experimentation, multi-round cross-stakeholder workshops, and an empirically grounded iterative evaluation framework—replacing traditional linear alignment with a closed-loop “intention–feedback–revision” cycle. Key contributions include: (1) establishing the first principled foundation for policy prototyping; (2) ensuring fidelity between collective stakeholder input and actual model behavior; and (3) demonstrating in an industrial AI lab that the approach significantly improves policy interpretability, intention fidelity, and cross-group consensus—thereby extending the methodological frontier of collaborative alignment. (149 words)

Current evaluations of AI governance proposals often fall into binary oppositions, overlooking implicit value trade-offs and lacking transparent analytical tools. This work proposes a multidimensional policy analysis framework that integrates expert interviews with computational text analysis to construct an interpretable scoring system across policy attributes, enabling cross-proposal comparison through visualization. Its novelty lies in three aspects: first, a multidimensional evaluation approach that avoids predetermined conclusions and explicitly reveals inherent trade-offs; second, a transparent hybrid methodology combining qualitative expert insights with quantitative computational validation; and third, the introduction of a domain-calibrated model as a benchmark against general-purpose large language models. The framework enables comparable, interpretable assessments of AI governance proposals across multiple attributes, allowing stakeholders to evaluate proposal relevance and coherence according to their own normative priorities.

This work addresses the inadequacy of existing large language model (LLM) lifecycle frameworks, which predominantly emphasize operational efficiency while lacking explicit support for security-critical activities—such as data provenance, component signing, and access control—and failing to align governance requirements with specific lifecycle phases. The paper proposes the first security-oriented LLM system lifecycle model, structured not by workflow but by security boundaries, organizing 32 phases into four layered pipelines: data, model, distribution, and application, while integrating LLMOps and governance pillars. It uniquely identifies 13 distinct security-critical phases and exposes a structural imbalance wherein regulatory evidence is concentrated at deployment despite pivotal decisions occurring during development. By mapping key standards—including NIST AI RMF, the EU AI Act, and ISO/IEC 42001—the study establishes a phase-to-governance correspondence mechanism, yielding a comprehensive, lifecycle-spanning security analysis framework that offers structured guidance for compliance and secure design.

This work addresses the challenges faced by large language model (LLM) agents in policy-constrained enterprise workflows—such as document auditing—including sparse feedback, performance degradation due to frequent rule updates, and the need to balance accuracy, reasoning cost, and auditability. To tackle these issues, the authors propose FRAMES, a framework that bootstraps deployable skills during cold-start and enables continuous skill evolution through consensus-based skill mutation, Pareto-optimal trade-offs between precision and computational cost, and a degradation-aware validation mechanism within a closed-loop system. FRAMES is the first approach to achieve auditable, efficient, and robust skill iteration in policy-intensive settings, demonstrating state-of-the-art precision–cost trade-offs on both an internal production system and the tau-bench benchmark, significantly outperforming existing baselines.