Towards Lightweight Reliability: Using Soft Prompts for Hallucination Mitigation in Large Language Models

2026-05-30

AI Summary

This work addresses the susceptibility of large language models to factual hallucinations in generative question answering, which undermines their reliability in high-stakes applications. To mitigate this issue, the authors propose RCSP, a parameter-efficient soft prompting method that uniquely integrates contrastive learning, curriculum learning, and KL divergence regularization within a composite loss function. This approach jointly optimizes the soft prompts to suppress hallucinations, encourage abstention under uncertainty, and preserve factual recall. Requiring only a minimal number of trainable parameters, RCSP significantly outperforms standard prompting baselines on Gemma-2 (12B) and Llama-3.1 (8B), achieving a superior F-score that effectively balances factual accuracy with system reliability.

Abstract

Large language models (LLMs) have seen widespread adoption across various domains, yet their reliability is frequently undermined by hallucinations - responses that are plausible-sounding but factually incorrect. In high-stakes domains, these errors can reduce trust and introduce real-world risk. To address this challenge, we present a parameter-efficient approach that uses soft prompts to mitigate hallucinated content and promote responsible abstention in generative question-answering (QA) tasks. Our method, called Responsible Contrastive Soft Prompting (RCSP), uses a composite loss to train soft prompts that balance three goals: suppressing hallucinatory content, encouraging abstention under uncertainty, and preserving or improving factual recall. To achieve these goals, we incorporate contrastive loss, curriculum learning, and KL regularization into our training mechanism. We evaluate our approach on five diverse generative QA datasets using an LLM-as-a-Judge framework. Experimental results on the Gemma 3 (12B) and Llama 3.1 (8B) backbones demonstrate that RCSP effectively balances factual recall with hallucination suppression and abstention, yielding a generally superior F-score over standard reasoning and instruction-based prompting baselines. Notably, these improvements are achieved by training only a fraction of the parameters required by other tuning techniques. Our results demonstrate that soft prompts provide a modular and computationally efficient path toward improving LLM reliability.

Problem

• hallucination
• large language models
• reliability
• generative QA
• responsible abstention

Innovation

• soft prompting
• hallucination mitigation
• parameter-efficient tuning
• responsible abstention
• contrastive learning