Vision Also You Need: Navigating Out-of-Distribution Detection with Multimodal Large Language Model
This work addresses the limitations of existing out-of-distribution (OOD) detection methods, which overly rely on textual features and struggle with distributional shifts in the visual domain—particularly under both near- and far-OOD scenarios. To overcome this, the authors propose MM-OOD, a novel framework that systematically leverages the multimodal reasoning and generative capabilities of large multimodal language models (MLLMs). For near-OOD samples, MM-OOD performs zero-shot inference by jointly utilizing image and text prompts. For far-OOD cases, it introduces a three-stage “sketch–generate–refine” pipeline that enhances multimodal prompting through generated visual exemplars. By moving beyond conventional unimodal or text-only paradigms, MM-OOD achieves state-of-the-art performance on multimodal benchmarks such as Food-101 and demonstrates strong scalability on ImageNet-1K.