When Hard Negatives Hurt: Bridging the Generative-Discriminative Gap in Hard Negative Synthesis for Retrieval

2026-05-31

AI Summary

This work addresses the limitations of traditional hard negative mining—such as insufficient corpus coverage, retriever scoring bias, and false positive interference—and the performance degradation often caused by negatives directly generated by large language models due to misalignment between generation and discrimination objectives. The authors propose CausalNeg, a novel framework that formally characterizes the generation-discrimination gap for the first time. It generates hard negatives of controllable difficulty through causal-guided counterfactual perturbations and mitigates source dependency via a query-perspective entropy maximization strategy. Integrating chain-of-thought reasoning, counterfactual data augmentation, and contrastive learning, CausalNeg enables interpretable, shortcut-free negative synthesis, significantly boosting retrieval performance across multiple benchmarks while effectively avoiding the performance drop commonly associated with generated negatives.

Abstract

Hard negative mining has become the dominant strategy for training retrievers, yet it faces intrinsic limitations: negatives are bounded by corpus availability, selected by retriever score rather than diagnostic value, and increasingly contaminated by false positives as the retriever improves. LLM-based synthesis offers a principled alternative, where negatives that are unconstrained, targeted, and free from false positive risk. But we show that naively incorporating generated negatives into contrastive learning often degrades retrieval performance. We identify and formalize the root cause as a generative-discriminative gap: LLM generation optimizes for fluent, plausible text, while contrastive learning demands strategic violations of relevance at the decision boundary. Our analysis reveals two compounding failure modes: discriminative-agnostic generation, where the LLM lacks an explicit model of query information needs and defaults to generic or topic-drifted text that provides no contrastive signal; and source-dependent shortcuts, where distributional artifacts enable the model to distinguish negatives by origin rather than relevance, causing gradient drift that actively corrupts optimization. To close this gap, we propose CausalNeg consisting of two main modules: (1) CoT-guided counterfactual perturbation for data construction: decomposes why a document satisfies a query into explicit information requirements, then surgically violates individual requirements to construct negatives with controlled, interpretable hardness. (2) Query-view entropy maximization during training: disperses generated negatives across the similarity spectrum, minimizing the mutual information between source identity and similarity scores to suppress shortcut exploitation. We make our code publicly available at https://github.com/mzhangzhicheng/CausalNeg.

Problem

• hard negative synthesis
• retrieval
• generative-discriminative gap
• contrastive learning
• false positives

Innovation

• hard negative synthesis
• generative-discriminative gap
• counterfactual perturbation
• contrastive learning
• query-view entropy maximization