hallucination detection

Existing evaluation metrics for generative models suffer from poor robustness and ill-defined fidelity-diversity trade-offs—structural deficiencies that hinder the practical deployment of synthetic data. Method: This paper introduces, for the first time, normative principles for synthetic data evaluation and an interpretable “sanity-check” suite. Through theoretical analysis and controlled experiments—including anomaly injection, distribution collapse, and scale perturbation—we systematically assess 12 mainstream metrics across three canonical failure modes. Contribution/Results: We demonstrate that all widely adopted metrics fail to meet practical reliability requirements. Based on these findings, we formulate a misuse warning checklist and a safety-guided usage protocol. Our work shifts the evaluation paradigm from “model-centric” to “metric-centric,” establishing a methodological foundation for trustworthy synthetic data assessment.

Current generative AI systems suffer from inadequate evaluation: static benchmarks fail to reflect real-world performance, and case-by-case audits lack scalability. To address this, we propose a scientific framework for evaluating generative AI—shifting assessment from fragmented auditing toward systematic, iterative, and institutionalized engineering practice. Our method introduces three foundational principles: (1) evaluation metrics must empirically capture real-world performance; (2) metrics must evolve dynamically over time; and (3) assessment must be conducted by specialized, accredited institutions operating under rigorous procedural standards. Drawing cross-domain insights from transportation, aerospace, and pharmaceutical safety regulation, we integrate systems safety engineering, metrology, and institutional design. The framework provides both theoretical grounding and actionable pathways for AI safety verification, regulatory alignment, and responsible industrial deployment.

This work addresses the challenge of efficiently and legally auditing high-risk language models at scale for harmful specialization—such as generation of child sexual abuse material (CSAM)—without producing illicit content. The authors propose a novel non-generative evaluation paradigm that detects harmful specialization by analyzing internal model states, specifically perturbations in intermediate representations induced by LoRA adapters. Leveraging Gaussian probing techniques, the method quantifies changes in internal representations through Gaussian latent ensembles. Experimental results demonstrate that this approach reliably distinguishes between benign and harmful models in CSAM-related tasks and exhibits robustness against adversarial interventions such as weight scaling.

This study addresses the systemic threat posed by generative AI–enabled multimodal disinformation to the information ecosystem, highlighting the lack of reproducibility and standardized benchmarks in current detection mechanisms. Through an initial longitudinal expert survey (N=21) involving AI researchers, policymakers, and disinformation specialists, the work synthesizes risk assessments of synthetic text, image, audio, and video content and evaluates the efficacy of existing mitigation strategies. Innovatively conceptualizing information integrity as critical infrastructure, the study proposes a response framework grounded in reproducible provenance standards and methodologies, advocating for standardized evaluation benchmarks and reproducibility checklists. Findings indicate that large-scale text generation presents greater systemic risk than deepfake videos, and experts broadly express skepticism toward purely technical detection approaches, favoring instead integrated governance solutions that combine provenance standards with regulatory frameworks.

Existing evaluation metrics for deep generative models (e.g., VAEs, GANs, diffusion models, Transformers) in engineering design—largely borrowed from statistical likelihood-based measures—fail to capture design-critical properties such as constraint satisfaction, functional performance, and design value. Method: We propose the first multidimensional evaluation framework tailored to engineering design, comprising four orthogonal dimensions: constraint compliance, functional effectiveness, novelty, and conditional controllability. We further develop an open-source, reproducible benchmark suite and software toolkit to bridge machine learning theory and design practice. Contribution/Results: The framework is rigorously validated on 2D visualization case studies and real-world engineering tasks—including bicycle frame and structural topology generation. Experiments demonstrate substantial improvements in alignment between automated evaluation and human-assessed design value: target achievement rate (+23.6%), geometric constraint compliance (+31.4%), and design novelty (+18.9%).

To address the performance bottleneck in industrial anomaly detection caused by the scarcity of real defective samples, this paper proposes a novel paradigm for efficient synthetic image utilization. Methodologically, we introduce a retrieval-based filtering mechanism that integrates text-guided image-to-image translation with content-aware image retrieval to identify highly relevant defective regions. Furthermore, we design a two-stage training framework: (1) low-cost rule-based pretraining—injecting noise or patches—and (2) retrieval-augmented fine-tuning of a diffusion model. This approach significantly enhances both the photorealism and task-specific fidelity of synthesized defect images. Evaluated on the MVTec AD benchmark, our method achieves state-of-the-art detection accuracy while reducing the cost of generating high-quality defect images by approximately 60%, thereby achieving a synergistic optimization of effectiveness and efficiency.

Generative AI datasets frequently suffer from opaque provenance, ambiguous ownership, unmitigated security and ethical risks, and loss of metadata and compliance information during sharing and reprocessing. Method: This paper proposes the first multidimensional, dynamic compliance assessment framework integrating transparency, accountability, and security. It introduces an open-source Python library grounded in data provenance techniques, incorporating metadata embedding, provenance graph modeling, and a configurable compliance rule engine to enable automated provenance tracking, real-time scoring, and compliance intervention across the full dataset lifecycle—acquisition, sharing, and repurposing. Contribution/Results: The framework has been seamlessly integrated into mainstream AI training and data processing pipelines, significantly enhancing the accountability and auditability of dataset construction and usage while ensuring robust regulatory adherence.

This work addresses the lack of reliable statistical evaluation methods for generative models, which hinders the assessment of their generalization performance and the estimability of evaluation metrics from finite samples. The authors propose a theoretical framework that systematically analyzes the conditions under which common evaluation metrics are statistically estimable, distinguishing between test-class-based metrics and divergence-based metrics in finite-sample settings. Leveraging tools from integral probability metrics (IPMs), Rényi divergences, and fat-shattering dimension, they rigorously establish—for the first time—that IPMs induced by bounded test classes admit arbitrarily accurate estimation from finite samples, whereas KL and Rényi divergences, which depend on rare events, do not. This study provides a foundational theoretical basis and practical guidance for evaluating generative models.

This study addresses the longstanding challenge of treating machine learning interpretability as a non-functional requirement lacking quantifiable metrics and validation mechanisms. To bridge this gap, the work proposes an innovative approach that reframes interpretability as a verifiable functional requirement through the integration of data and model provenance. By synergizing principles from requirements engineering and machine learning engineering, the authors develop a systematic and operational verification framework. This framework enables, for the first time, the explicit specification and empirical validation of interpretability requirements, thereby substantially enhancing the engineering rigor and trustworthiness of machine learning system development.

This work addresses the limited effectiveness and diversity of generative test inputs in boundary testing of deep learning image classifiers by proposing a latent-space truncation regularization method based on StyleGAN. By integrating latent code mixing with a bisection-search-optimized truncation strategy, the approach efficiently generates high-quality and diverse boundary test samples in the latent space. Experimental results on MNIST, Fashion-MNIST, and CIFAR-10 demonstrate that the proposed method significantly outperforms random truncation, achieving higher fault detection rates while simultaneously enhancing both the validity and diversity of the generated test inputs.