Tiered Agentic Oversight: A Hierarchical Multi-Agent System for AI Safety in Healthcare

2025-06-14

AI Summary

To address the weak error-detection capability and high single-point-failure risk of large language models (LLMs) in clinical applications, this work proposes a hierarchical multi-agent system that emulates the clinical hierarchy—nurse → general physician → specialist—establishing an adaptive, tiered supervision architecture. We innovatively introduce task-complexity-driven dynamic routing, role-aware cross-tier collaboration, and a high-capability-LLM-first deployment strategy. Role-playing and ablation-driven LLM resource optimization are grounded in clinical workflows. Experiments demonstrate state-of-the-art (SOTA) performance on 4 of 5 medical safety benchmarks, with up to 8.2% absolute improvement; safety increases by 3.2% over flat single-layer systems; and triage accuracy rises from 40% to 60% when incorporating physician feedback. Crucially, we empirically identify Tier-1 agents as critical safety gatekeepers—a novel finding in clinical AI safety.

Technology Category

• Multiagent Systems: Multiagent Learning
• Natural Language Processing: Safety and Robustness
• Machine Learning: Large Multimodal Models (LMMs)

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Current large language models (LLMs), despite their power, can introduce safety risks in clinical settings due to limitations such as poor error detection and single point of failure. To address this, we propose Tiered Agentic Oversight (TAO), a hierarchical multi-agent framework that enhances AI safety through layered, automated supervision. Inspired by clinical hierarchies (e.g., nurse, physician, specialist), TAO conducts agent routing based on task complexity and agent roles. Leveraging automated inter- and intra-tier collaboration and role-playing, TAO creates a robust safety framework. Ablation studies reveal that TAO's superior performance is driven by its adaptive tiered architecture, which improves safety by over 3.2% compared to static single-tier configurations; the critical role of its lower tiers, particularly tier 1, whose removal most significantly impacts safety; and the strategic assignment of more advanced LLM to these initial tiers, which boosts performance by over 2% compared to less optimal allocations while achieving near-peak safety efficiently. These mechanisms enable TAO to outperform single-agent and multi-agent frameworks in 4 out of 5 healthcare safety benchmarks, showing up to an 8.2% improvement over the next-best methods in these evaluations. Finally, we validate TAO via an auxiliary clinician-in-the-loop study where integrating expert feedback improved TAO's accuracy in medical triage from 40% to 60%.

Problem

• Enhances AI safety in healthcare using hierarchical multi-agent system
• Reduces clinical risks from LLM errors via tiered oversight
• Improves medical triage accuracy through adaptive agent collaboration

Innovation

• Hierarchical multi-agent system for AI safety
• Automated inter-tier and intra-tier collaboration
• Adaptive tiered architecture boosts safety