Your Finetuned Large Language Model is Already a Powerful Out-of-distribution Detector

2024-04-07

arXiv.org

AI Summary

To address the insufficient robustness of large language models (LLMs) against out-of-distribution (OOD) inputs in question-answering systems, this paper proposes a training-free, plug-and-play likelihood ratio detection method. It computes the conditional likelihood ratio between a pre-trained LLM and its fine-tuned counterpart on the same input question as an OOD score. We empirically discover, for the first time, that fine-tuned LLMs inherently possess discriminative capability for OOD detection. Unlike conventional approaches requiring auxiliary modules or dedicated OOD training, our method relies solely on forward inference and lightweight likelihood comparisons—entailing zero gradient computation and zero training overhead. Extensive evaluation across distant/near OOD identification, spam question detection, and multi-source QA benchmarks demonstrates consistent superiority over state-of-the-art methods. The implementation is open-sourced and supports one-click deployment via Hugging Face models.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Question Answering
• Data Mining & Knowledge Management: Anomaly/Outlier Detection

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

We revisit the likelihood ratio between a pretrained large language model (LLM) and its finetuned variant as a criterion for out-of-distribution (OOD) detection. The intuition behind such a criterion is that, the pretrained LLM has the prior knowledge about OOD data due to its large amount of training data, and once finetuned with the in-distribution data, the LLM has sufficient knowledge to distinguish their difference. Leveraging the power of LLMs, we show that, the likelihood ratio can serve as an effective OOD detection criterion. Moreover, we apply the proposed LLM-based likelihood ratio to detect OOD questions in question-answering (QA) systems, which can be used to improve the performance of specialized LLMs for general questions. Given that likelihood can be easily obtained by the loss functions within contemporary neural network frameworks, it is straightforward to implement this approach in practice. Since both the pretrained LLMs and its various finetuned models are widely available from online platforms such as Hugging Face, our proposed criterion can be effortlessly incorporated for OOD detection without the need for further training. We conduct comprehensive evaluation across on multiple settings, including far OOD, near OOD, spam detection, and QA scenarios, to demonstrate the effectiveness of the method. Code can be found at https://github.com/andiac/LLMOODratio

Problem

• Detects out-of-distribution data using likelihood ratio.
• Improves question-answering systems by identifying OOD questions.
• Utilizes pretrained and finetuned LLMs without additional training.

Innovation

• Leverages pretrained and finetuned LLMs for OOD detection
• Uses likelihood ratio as effective OOD detection criterion
• Applies method to QA systems for improved performance