hard negative mining

Diffusion models (DMs) suffer from insufficient prompt adherence in text-to-image generation; existing negative prompting approaches rely on handcrafted prompts, incur information loss, and exhibit poor generalization. This paper establishes, for the first time, an intrinsic connection between negative prompting and classifier-free guidance (CFG), and proposes a training-free, resource-free adaptive negative sampling method: it implicitly models negation semantics via the CFG mechanism and dynamically aligns with negative concepts during sampling—thereby enhancing fidelity to the positive prompt. Crucially, the method eliminates explicit negative prompt inputs entirely, offering universal applicability and plug-and-play deployment. Extensive evaluations across multiple benchmarks demonstrate substantial improvements over strong baselines; in human preference studies, our method is selected twice as often as competing approaches.

The role of negative examples—such as incorrect, suboptimal, or semantically plausible-but-wrong (“near-miss”) responses—in large language model (LLM) alignment training remains poorly understood. Method: We propose a likelihood-ratio (Likra) framework to systematically quantify the impact of negative examples across supervised fine-tuning (SFT) and reinforcement learning from human feedback (RLHF)/direct preference optimization (DPO) stages, using multiple-choice QA benchmarks. Contribution/Results: We uncover, for the first time, that negative examples—especially near-miss ones—induce abrupt, step-like improvements in learning curves; yield significantly higher per-sample gains than positive-only SFT; and are indispensable for suppressing hallucination and reducing the probability of generating plausible-but-incorrect outputs. These findings establish the critical causal attribution of negative examples in LLM alignment, providing both a novel paradigm and empirical grounding for efficient, robust alignment strategies.

This study systematically investigates, for the first time, the impact of negative sampling strategies on unsupervised neural topic modeling based on variational autoencoders (VAEs). To address the limited discriminative capacity of conventional VAE decoders, we propose integrating a “learned contrastive” mechanism into the decoding process, explicitly modeling positive and negative word–topic associations. Our approach significantly improves topic quality: on four benchmark datasets, normalized pointwise mutual information (NPMI) increases by up to 12.7%, topic diversity rises by 9.3%, and document classification accuracy improves by an average of 5.1%. Comprehensive evaluation—combining automatic metrics (NPMI, diversity) and human assessment—confirms that negative sampling critically enhances semantic discriminability and structural interpretability. This work establishes a novel paradigm for unsupervised topic modeling and demonstrates an effective pathway for transferring contrastive learning into purely generative frameworks.

In implicit collaborative filtering, existing negative sampling strategies tend to oversample in dense regions of user-item interactions, leading to homogeneous negative instances that impair model expressiveness and generalization. To address this, we propose DivNS—a novel negative sampling framework that explicitly models diversity for the first time. DivNS comprises three stages: user-specific cache management, diversity-aware sampling, and synthetic negative instance generation—collectively enhancing coverage breadth and discriminative power of negative samples. It requires no additional annotations and seamlessly integrates with mainstream implicit CF models. Extensive experiments on four public benchmarks demonstrate that DivNS consistently improves Recall@K and NDCG@K by an average of 3.2% over strong baselines—including BPR, IPS, and SLIME—while maintaining computationally efficient overhead.

Large language models (LLMs) exhibit systematic negative bias in binary decision tasks requiring complex reasoning, leading to significant precision–recall imbalance. This paper introduces the Negative Attention Score (NAS) — the first quantitative metric for measuring such bias — and identifies the attention heads predominantly responsible for it. We then propose NASA, a parameter-efficient fine-tuning method based on LoRA-style adapters, enabling targeted calibration of these “negative-biased” heads. Our approach integrates attention mechanism analysis, NAS-based modeling, and a multi-domain reasoning evaluation framework spanning mathematical, commonsense, and symbolic reasoning tasks. NASA substantially reduces the precision–recall gap while preserving or improving overall accuracy and cross-task generalization. Key contributions include: (1) a formal, systematic definition of NAS; (2) interpretable localization of bias-inducing attention heads; and (3) a lightweight, transferable paradigm for targeted bias mitigation.

This work addresses the limitations of traditional hard negative mining—such as insufficient corpus coverage, retriever scoring bias, and false positive interference—and the performance degradation often caused by negatives directly generated by large language models due to misalignment between generation and discrimination objectives. The authors propose CausalNeg, a novel framework that formally characterizes the generation-discrimination gap for the first time. It generates hard negatives of controllable difficulty through causal-guided counterfactual perturbations and mitigates source dependency via a query-perspective entropy maximization strategy. Integrating chain-of-thought reasoning, counterfactual data augmentation, and contrastive learning, CausalNeg enables interpretable, shortcut-free negative synthesis, significantly boosting retrieval performance across multiple benchmarks while effectively avoiding the performance drop commonly associated with generated negatives.

This work addresses the limitations of conventional negative sampling strategies in two-tower model training, which often yield easy negatives that hinder discriminative learning and exacerbate popularity bias and feedback loops. To overcome these issues, the authors propose the first large language model (LLM)-based framework for real-time hard negative sampling. During training, the LLM performs semantic clustering and dynamically generates challenging yet relevant negatives from within semantically similar clusters. This approach significantly enhances the representation learning capability of two-tower models, effectively mitigates popularity bias, and breaks detrimental feedback cycles—all while maintaining low computational overhead. Extensive experiments demonstrate that the proposed framework consistently outperforms state-of-the-art industrial negative sampling methods on both public benchmarks and a billion-scale production system, yielding substantial gains in retrieval performance.

Existing knowledge distillation approaches overemphasize hard negative samples while neglecting the overall distribution of teacher model output scores, thereby limiting the student model’s ability to learn a complete preference structure and impairing its generalization. This work is the first to systematically highlight the importance of preserving the teacher’s score distribution and proposes a hierarchical sampling strategy that uniformly covers the full range of teacher-assigned scores during training. By effectively maintaining the variance and entropy of the teacher’s outputs, this method enables more comprehensive transfer of preference information. It overcomes the limitations of conventional approaches that rely solely on hard negatives and achieves significant performance gains over Top-K and random sampling baselines on both in-domain and out-of-domain dense retrieval benchmarks, demonstrating that sustaining the diversity of teacher scores is crucial for enhancing student model performance.

Existing graph contrastive learning methods rely on static negative sampling, which struggles to dynamically balance informativeness and computational overhead. This work proposes AdNGCL, a novel framework that introduces, for the first time, a budget-aware, loss-sensitive Hardness-Aware Negative Scheduler (HANS). HANS formulates negative sample selection as a dynamic process governed by loss gating and computational budget constraints, adaptively adjusting sampling strides across hard, medium, and easy negatives while periodically refreshing the pool to preserve diversity. Evaluated on nine benchmark graph datasets, AdNGCL achieves state-of-the-art performance on seven and runner-up results on two, significantly improving accuracy while enabling explicit control over computational cost.