Auditing Reproducibility in Non-Targeted Analysis: 103 LC/GC--HRMS Tools Reveal Temporal Divergence Between Openness and Operability

2025-12-23

AI Summary

Despite growing adoption of non-targeted analysis (NTA) in food safety, systematic evaluation of liquid/gas chromatography–high-resolution mass spectrometry (LC/GC-HRMS) NTA tools against FAIR principles and the BP4NTA operational pillars—laboratory validation, data/code availability, standardized formats, knowledge integration, and portable implementation—remains lacking. Method: We conducted a longitudinal, systematic audit of 103 NTA tools published between 2004 and 2025, assessing compliance across all six BP4NTA pillars. Contribution/Results: We found a marked increase in openness (56% → 86%) but a paradoxical decline in practical reproducibility (55% → 43%), quantifying for the first time the persistent “discoverable but not runnable” gap. Critical synergistic deficits between Pillar C1 (validation) and C6 (portable implementation) emerged as the primary bottleneck for regulatory-grade reproducibility. This study fills a key gap in food safety NTA tool assessment and proposes a multidimensional audit framework; only 17% of tools satisfy both validation and portability criteria—providing empirical grounding and actionable pathways toward fully reproducible NTA workflows.

Technology Category

• Natural Language Processing: Safety and Robustness
• Search and Optimization: Evaluation and Analysis
• Machine Learning: Evaluation and Analysis

Application Category

• Security and Privacy: Data transparency and provenance
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Web Mining and Content Analysis: Web data provenance, reliability, and authenticity

Abstract

In 2008, melamine in infant formula forced laboratories across three continents to verify a compound they had never monitored. Non-targeted analysis using LC/GC-HRMS handles these cases. But when findings trigger regulatory action, reproducibility becomes operational: can an independent laboratory repeat the analysis and reach the same conclusion? We assessed 103 tools (2004-2025) against six pillars drawn from FAIR and BP4NTA principles: laboratory validation (C1), data availability (C2), code availability (C3), standardised formats (C4), knowledge integration (C5), and portable implementation (C6). Health contributed 51 tools, Pharma 31, and Chemistry 21. Nine in ten tools shared data (C2, 90/103, 87%). Fewer than four in ten supported portable implementations (C6, 40/103, 39%). Validation and portability rarely appeared together (C1+C6, 18/103, 17%). Over twenty-one years, openness climbed from 56% to 86% while operability dropped from 55% to 43%. No tool addressed food safety. Journal data-sharing policies increased what authors share but not what reviewers can run. Tools became easier to find but harder to execute. Strengthening C1, C4, and C6 would turn documented artifacts into workflows that external laboratories can replay.

Problem

• Auditing reproducibility of non-targeted analysis tools
• Assessing divergence between data openness and operational portability
• Identifying gaps in validation, standardization, and implementation

Innovation

• Assessed 103 LC/GC-HRMS tools using six FAIR and BP4NTA pillars
• Found openness increased but operability decreased over twenty-one years
• Proposed strengthening validation, formats, and portability for reproducibility