Knowledge before Reasoning: EC-Reason-Bench, a Training-Free Diagnostic Benchmark for LLM Enzyme Classification

2026-07-28

AI Summary

General-purpose large language models (LLMs) struggle to accurately predict full EC numbers in enzyme classification, with near-zero accuracy on levels two through four. This work proposes EC-Reason-Bench, the first training-free diagnostic evaluation protocol that decomposes enzyme classification capability into four orthogonal dimensions: output structure, external knowledge, reasoning architecture, and reasoning robustness. Systematic zero-shot evaluations—employing open- and closed-book settings, chain-of-thought reasoning, cascaded prediction, and nearest-neighbor EC voting—reveal that external knowledge must be incorporated prior to reasoning, which primarily arbitrates conflicting evidence. Integrating external knowledge substantially boosts performance; the best LLM configuration matches nearest-neighbor voting overall but outperforms it under adversarial evidence, while degrading significantly on multifunctional enzymes. These findings demonstrate that a single metric is insufficient for comprehensive performance assessment.

Abstract

Enzyme function prediction is a hierarchical, knowledge-intensive form of protein function classification. Existing benchmarks expose an anomaly: general LLMs often get the coarse first level right, yet once asked for a complete EC number their accuracy at levels two through four drops to almost zero, while specialized models and tools stay usable. We propose EC-Reason-Bench, a training-free, diagnostic evaluation protocol built to answer two questions: why general LLMs score close to nothing on EC number prediction, and how much of that loss can be recovered without updating a single weight. We break enzyme classification ability into four orthogonal levers that can each be measured on their own: output structure, external knowledge, reasoning structure, and reasoning robustness. We test each lever with an inference-time method against a shared zero-shot baseline reproducing previously reported near-zero performance. Experiments with several strong reasoning LLMs yield four main findings. First, external knowledge is decisive and must precede reasoning: uniformly low closed-book performance rises sharply with open-book access, narrowing model gaps. Second, in closed-book settings, whether cascading and chain-of-thought help or hurt depends on a model's tendency to abstain. Third, once evidence is available the aggregate score of the best LLM setting is indistinguishable from simply voting the EC numbers of the nearest retrieved neighbors; that tie is an artifact of averaging, and it hides a large gain on adversarial evidence set against an equally large loss on multi-functional enzymes. Reasoning over evidence therefore acts as an arbiter of conflicting neighbors rather than as a source of knowledge, and no single-number leaderboard can see it. Fourth, accuracy obeys a law of homology availability.

Problem

• enzyme classification
• EC number prediction
• large language models
• protein function
• hierarchical classification

Innovation

• training-free benchmark
• enzyme classification
• external knowledge integration
• reasoning robustness
• EC number prediction