Levels of Autonomy for AI Agents

2025-06-14

AI Summary

How to rigorously define the autonomy level of AI agents while balancing innovation potential and risk mitigation? This paper proposes the first systematic, quantifiable five-level autonomy framework, explicitly treating autonomy as a design dimension orthogonal to capability and environment, and delineating control boundaries based on user roles (from operator to observer). It introduces the novel concept of an “AI Autonomy Certificate” to enable tiered governance for both single and multi-agent systems, and establishes a new evaluation paradigm centered on human–agent interaction modalities. Integrating human–agent modeling, hierarchical design principles, governance architecture, and behavioral norms, the framework yields a calibrated, verifiable, and implementable methodology for autonomy assessment. It provides a technical pathway for developing safe, controllable AI agents and delivers an auditable certification basis for regulatory oversight.

Technology Category

• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Multiagent Systems: Agent/AI Theories and Architectures
• Cognitive Modeling & Cognitive Systems: Agent Architectures

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search

Abstract

Autonomy is a double-edged sword for AI agents, simultaneously unlocking transformative possibilities and serious risks. How can agent developers calibrate the appropriate levels of autonomy at which their agents should operate? We argue that an agent's level of autonomy can be treated as a deliberate design decision, separate from its capability and operational environment. In this work, we define five levels of escalating agent autonomy, characterized by the roles a user can take when interacting with an agent: operator, collaborator, consultant, approver, and observer. Within each level, we describe the ways by which a user can exert control over the agent and open questions for how to design the nature of user-agent interaction. We then highlight a potential application of our framework towards AI autonomy certificates to govern agent behavior in single- and multi-agent systems. We conclude by proposing early ideas for evaluating agents' autonomy. Our work aims to contribute meaningful, practical steps towards responsibly deployed and useful AI agents in the real world.

Problem

• Determine appropriate autonomy levels for AI agents
• Define five escalating autonomy levels for user-agent interaction
• Propose framework for AI autonomy certificates and evaluation

Innovation

• Defines five escalating AI autonomy levels
• Proposes AI autonomy certificates framework
• Suggests early autonomy evaluation methods