Physically Verifiable Evidence and LLM-Based Reporting for Bearing Fault Diagnosis

2026-07-24

AI Summary

This work addresses the lack of verifiability in existing AI-based bearing fault diagnosis methods and the hallucination issues prevalent in large language model (LLM)-generated diagnostic reports. The authors propose the Diagnostic Evidence Network (DENet), which leverages multi-task learning to simultaneously predict fault categories, theoretically interpretable characteristic frequencies, and temporal locations of impulsive signals. DENet introduces the first label-free, inference-time verification mechanism that detects misclassifications by measuring deviations between predicted and theoretical characteristic frequencies. To ensure factual reporting, a QLoRA-fine-tuned constrained LLM is employed solely to translate structured diagnostic evidence into natural language, thereby eliminating fabricated content. Experimental results demonstrate that DENet maintains high diagnostic accuracy while achieving an AUROC of 0.970 for misclassification detection and reduces unsubstantiated claims in LLM-generated reports from 10–12% to 2%.

Abstract

Trustworthy deployment of AI-based diagnosis in safety-critical mechanical systems hinges on validation: whether a prediction can be checked against physical reality before it is acted upon. Current intelligent fault diagnosers fail this standard in two ways. Their standard output, a class label with a softmax confidence score, is an internal statistic of the classifier, offering nothing checkable against independent physical knowledge; and the growing use of generative language models in maintenance reporting adds a second risk: hallucinated content entering reports on which decisions rest. Taking bearing fault diagnosis as the testbed, this work addresses both problems from the output side. The proposed Diagnostic Evidence Network (DENet) is an encoder-agnostic multi-task framework extending the output to a structured evidence record: the classification, a predicted characteristic frequency comparable against the theoretical value determined by bearing geometry and shaft speed, and a temporal localization of transient impulses inspectable on the raw waveform. Across four encoders and three public datasets, this evidence incurs no statistically significant accuracy cost, with a frequency error of about 6 Hz on 1,024-point segments where spectral estimation is structurally inapplicable. Centrally, the deviation between predicted and theoretical frequency constitutes a label-free, inference-time validation signal: it detects misclassifications with AUROC values of 0.970 and 0.871, and remains discriminative in the high-confidence regime where confidence-derived detectors are blind. Finally, a QLoRA-adapted language model is constrained to translate, but never generate, diagnostic content, reducing unsupported-claim rates from 10-12% to 2% and eliminating fabricated quantities.

Problem

• bearing fault diagnosis
• physical verifiability
• hallucination
• trustworthy AI
• diagnostic validation

Innovation

• Diagnostic Evidence Network
• physically verifiable evidence
• bearing fault diagnosis
• hallucination mitigation
• inference-time validation