ASK-NN: An Asymmetric Nearest-Neighbor Test that detects Distribution Drifts in Natural Language

2026-07-17

AI Summary

This work addresses hallucination and artificial text generation in large language models caused by distributional shifts between prompt and response hidden states, particularly in scenarios with asymmetric reference-query sequence lengths. To tackle this, the authors propose ASK-NN, an asymmetric two-sample test based on directed k-nearest neighbor graphs. The method detects distributional drift by statistically counting how often reference points retain other reference points as nearest neighbors within a mixed sample. ASK-NN is the first to introduce asymmetric nearest neighbor graphs for natural language distribution testing, offering exact conditional mean and variance under finite samples and establishing asymptotic normality and consistency under fixed alternative hypotheses. Experiments demonstrate that ASK-NN matches or outperforms kernel- and graph-based baselines on synthetic benchmarks, artificial text detection, and token-level hidden state hallucination tasks, while remaining computationally efficient and straightforward to implement.

Abstract

Hallucinations and artificial text in LLM-generated outputs often appear as distributional deviations between prompt and response hidden-state distributions. Since prompts or retrieved contexts typically serve as reference samples and responses as query samples, with major differences in length, these asymmetries motivate the use of change test statistics that treat the two samples differently. We consider an asymmetric two-sample test ASK-NN based on the directed k-nearest-neighbor graph. Our statistic counts reference points whose nearest neighbor in the pooled sample is also a reference point. Under the permutation null, it admits an exact finite-sample conditional mean and variance; we further establish asymptotic normality and consistency under fixed alternatives. ASK-NN is computationally effective and easy to implement. Empirically, it is competitive with kernel and graph-based baselines on synthetic benchmarks, artificial-text detection, and LLM hallucination detection from token-level hidden states.

Problem

• distribution drift
• asymmetric nearest-neighbor
• LLM hallucination
• artificial text detection
• hidden-state distribution

Innovation

• asymmetric nearest-neighbor test
• distribution drift detection
• large language model hallucination
• directed k-nearest-neighbor graph
• two-sample testing