🤖 AI Summary
This study addresses the lack of consistent verification for faults and low-precision behaviors in LLM-modernized scientific software by proposing a novel differential fault injection framework. The approach systematically evaluates transformation correctness and robustness by injecting identical deterministic faults into both original and LLM-generated code, combined with GAMESS-driven instrumentation and contraction model prediction. Validated through over 2,200 runs, the fault absorption model demonstrates high accuracy and reliability. Furthermore, 200 paired injection experiments yielded fully consistent results, successfully exposing and rectifying deep-seated synchronization defects such as parallel deadlocks and false convergence. These findings effectively ensure the trustworthiness of modernizing legacy scientific codebases using large language models, providing a rigorous methodology for validating behavioral equivalence under faulty conditions.
📝 Abstract
Large language model (LLM) agents are increasingly used to modernize the legacy Fortran underlying production scientific software, but validation of these transformations emphasizes nominal executions and may not test whether a modernization preserves the original code's response to faults, perturbations, and reduced precision. We present a differential fault-injection validation method: a harness instruments the shared self-consistent-field driver of GAMESS at twelve sites and applies identical, deterministic faults to the original and LLM-modernized implementations, isolating the converted integral kernels. Across more than 2,200 runs, transient-fault absorption costs match a contraction-based model (predicted slopes 0.74 and 1.49 iterations per bit; measured 0.82 and 1.50), persistent perturbations halve final-energy error per additional bit, and the campaigns expose phase-dependent parallel deadlocks and false convergence under reduced precision. The original and modernized kernels agree in all 200 paired injections, and a measurement-guided synchronization change composes with the modernization, matching in all 40 pairs.