🤖 AI Summary
This work addresses the lack of verifiable and traceable mechanisms for assessing biological novelty in current large language model–based scientific agents, which undermines scientific validity. The authors propose Plato-Bio, a system that integrates explicit workflow states, claim–evidence linking, citation verification, scoped file writing, and publication gating into the Plato/Denario architecture. For the first time, it embeds software contracts and auditable benchmarks into the biological discovery pipeline, mitigating three critical evaluation flaws: task-domain drift, missing methodological signals, and incomplete evidence denominators. Built on a Python workflow engine, Plato-Bio incorporates TF-IDF ranking, AlphaFold structural alignment, RMSD computation, and confidence masking. Evaluations demonstrate accurate retrospective rediscovery of the fish oil–Raynaud’s phenomenon link, Cα-RMSD values below 1 Å for 11 of 15 proteins, 27 traceable unvalidated hypothesis regions, and 100% test suite pass rates.
📝 Abstract
Large language model research agents can connect literature retrieval, analysis code, and manuscript preparation, but coherent output does not establish scientific validity. We developed Plato-Bio, a biology-routed extension of the open Plato/Denario architecture that couples explicit workflow states with provenance records, citation checks, claim-to-evidence links, scoped file writes, and publication gates. A source audit identified and repaired three defects that could distort evaluation: loss of task domain in the default factory, omission of declared method signals from scoring, and evidence sidecars that lacked the drafted-claim denominator. On the current clean revision, the full Python suite completed with 931 passes, six skips, and no failures or errors; targeted biology, genomics, evidence/citation, and adversarial-safety suites likewise completed without failure. We evaluated two narrow use cases. In a frozen historical rediscovery task, independent pre-1986 literature bridges ranked the later-studied relation between fish oil and Raynaud phenomenon first; TF-IDF ranked it second and corpus frequency third. This single curated task measures retrospective ranking, not prospective discovery. In a separate comparison of AlphaFold models with experimental structures for 15 human proteins, 11 targets had high-confidence-core C-alpha RMSD below 1 Angstrom (median 0.501 Angstrom). Four targets exceeded 2 Angstrom, and confidence masking reduced the SUMO1 discrepancy from 16.61 to 2.58 Angstrom over 74 residues. The workflow emitted 27 traceable discrepancy regions, all retained as unvalidated hypotheses. Plato-Bio therefore provides reproducible software contracts and auditable screening baselines; broader claims of agent efficacy or biological novelty require preregistered evaluation, independent review, and prospective validation.