CockpitHAT: Dependency-Graph-Driven Hierarchical Attribution for Embodied Multi-Agent Cockpits

2026-08-03

AI Summary

This work addresses the challenge of detecting high-risk physical operation failures in embodied multi-agent systems powered by large language models—such as those in automotive cockpits—where "correctness collapse" renders textual execution traces insufficient for fault diagnosis. To this end, the authors propose a hierarchical attribution framework that replaces conventional positional windows with distance thresholds derived from interaction dependency graphs, integrates multimodal embodied evidence, and incorporates a safety-enhancement mechanism within a confidence-weighted expert consensus. This approach uniquely unifies dependency graph structures, multimodal embodied signals, and safety-aware risk calibration to enable reliable fault diagnosis in real-world embodied systems. Evaluated on the Who&When benchmark, the method outperforms the state-of-the-art ECHO by 17.6 and 16.7 percentage points, and achieves 78.3% agent-level and 38.2% step-level attribution accuracy on the newly introduced CockpitBench.

Abstract

LLM multi-agent systems suffer from Correctness Collapse, where high task-level accuracy conceals severe process-level failures. This is especially hazardous in safety-critical embodied settings such as automotive cockpits, where lexically correct utterances may trigger dangerous physical operations. Existing attribution methods rely on text traces alone, missing dependency structure, multi-channel evidence, and safety-aware evaluation. We introduce CockpitHAT, a hierarchical attribution framework that replaces positional windows with dependency-distance thresholds from interaction DAGs, integrates multi-channel evidence via an embodied adapter, and applies a safety-uplift to high-risk failures during confidence-weighted analyst consensus. We further release CockpitBench, a benchmark of 212 annotated failure traces spanning dialogue, vehicle-state, environmental, and memory channels, each labeled with ISO 26262 ASIL severity via three-expert consensus. On the public Who&When benchmark, CockpitHAT achieves agent-level / step-exact accuracies of 77.9% / 37.8% on the Hand-Crafted split and 86.5% / 46.0% on the Algorithm-Generated split, surpassing the text-only SOTA ECHO by up to 17.6 / 16.7 points. On CockpitBench, it attains 78.3% agent-level and 38.2% step-exact accuracy. These results establish dependency-aware, multi-channel, risk-calibrated attribution as an effective paradigm for reliable failure diagnosis in real-world embodied LLM multi-agent systems.

Problem

• Correctness Collapse
• embodied multi-agent systems
• safety-critical attribution
• dependency structure
• multi-channel evidence

Innovation

• dependency-graph-driven attribution
• embodied multi-agent systems
• multi-channel evidence fusion
• safety-aware failure diagnosis
• hierarchical attribution