SABRE: A Multi-Agent Approach for Selecting Out-of-Distribution Detectors Under a Budget

2026-08-03

AI Summary

This work addresses the instability and reliability inversion often observed in existing out-of-distribution (OOD) detectors based on fixed references when deployed across domains. To overcome this limitation, the authors propose SABRE, a novel framework that introduces, for the first time, a multi-agent collaborative mechanism to dynamically select and fuse multiple posterior OOD detectors during inference, enabling adaptive and reliable detection without prior knowledge of the target domain. SABRE comprises three language model agents—Selector, Reporter, and Analyst—and integrates four new multimodal density-based detectors, few-shot in-domain calibration, and an evidence aggregation strategy to accurately identify the best-performing detector per domain under a limited query budget. Experiments demonstrate that SABRE restores robust performance in scenarios where conventional methods fail, and ablation studies confirm the complementary contributions of each agent.

Abstract

Post-hoc out-of-distribution (OOD) detection for vision-language models assumes that a detector chosen on a benchmark stays reliable once deployed. We show this fails across domains: on a single frozen encoder, a detector that leads in one domain can invert in another, scoring in-distribution inputs as more anomalous than genuine outliers, and the best detector changes from domain to domain, so no fixed choice is reliable throughout. We introduce SABRE (Selective Agentic Budgeted Reliability Ensemble,) which replaces this fixed choice with per-regime selection at inference. Three language-model agents reason over a library of post-hoc detectors under a bounded query budget: a Selector chooses which detector to consult next, a Reporter consolidates the evidence for each input, and an Analyst calibrates detector reliability on a small labeled sample held out from the deployment domain and disjoint from the test data, weighting selection and aggregation without ever observing a scored input's label. The library includes four multimodal density detectors we propose. Inferring the operating regime from data, SABRE tracks the strongest detector in each domain without prior knowledge of it, recovering reliable detection where a conventional detector inverts and converging to that detector where it is sound. A component analysis shows the agents are complementary: the Reporter's feedback yields consistent gains, and the Analyst's calibration is decisive against inversion, ruling out unreliable detectors so that aggregation no longer cancels the sound ones. Since no fixed rule can be trusted across domains, reliability must be established at deployment rather than assumed from a benchmark, and SABRE shows this can be done automatically.

Problem

• out-of-distribution detection
• vision-language models
• domain shift
• detector reliability
• post-hoc detection

Innovation

• multi-agent system
• out-of-distribution detection
• dynamic detector selection
• vision-language models
• reliability calibration