Hallucination Detection and Mitigation in Scientific Text Simplification using Ensemble Approaches: DS@GT at CLEF 2025 SimpleText

2025-08-15

AI Summary

This study addresses the pervasive hallucination generation and information distortion in scientific text simplification. Methodologically, it proposes a multi-source signal fusion framework for integrated hallucination detection and faithfulness-preserving correction. Specifically, it constructs a meta-learning-based ensemble classifier by jointly leveraging BERT-based classification, semantic similarity metrics, natural language inference (NLI) models, and large language model (LLM) reasoning capabilities to enhance hallucination detection robustness. Furthermore, it introduces an LLM-based post-editing mechanism explicitly constrained by the original text to ensure faithful reconstruction of simplified outputs. Evaluated on the CLEF 2025 SimpleText Task 2 benchmark, the framework achieves significant improvements in both hallucination detection accuracy and output credibility. It establishes a novel paradigm for scientific text simplification that effectively balances creativity with factual fidelity.

Technology Category

• Natural Language Processing: Fact-Checking / Misinformation Detection (NLP Focus)
• Search and Optimization: Metareasoning and Metaheuristics
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Web Mining and Content Analysis: Normalization, clustering, classification, and summarization of Web text

Abstract

In this paper, we describe our methodology for the CLEF 2025 SimpleText Task 2, which focuses on detecting and evaluating creative generation and information distortion in scientific text simplification. Our solution integrates multiple strategies: we construct an ensemble framework that leverages BERT-based classifier, semantic similarity measure, natural language inference model, and large language model (LLM) reasoning. These diverse signals are combined using meta-classifiers to enhance the robustness of spurious and distortion detection. Additionally, for grounded generation, we employ an LLM-based post-editing system that revises simplifications based on the original input texts.

Problem

• Detect hallucinations in scientific text simplification
• Evaluate information distortion in simplified texts
• Mitigate spurious content using ensemble approaches

Innovation

• Ensemble framework with BERT and semantic similarity
• Meta-classifiers combine diverse signals for robustness
• LLM-based post-editing system for grounded generation