DN-Hypo-Pipeline: An AI-Driven Workflow for Hypothesis Generation via Large Language Models and Scientific Explanations

2026-06-07

AI Summary

This work proposes a theory-guided hypothesis generation framework that integrates scientific explanatory mechanisms with large language models (LLMs) to systematically derive novel, testable hypotheses from scientific literature. Addressing the limitations of traditional approaches—which struggle to efficiently and rigorously generate hypotheses from existing research—the method begins with published conclusions, reconstructs their underlying theoretical foundations, and produces structured new hypotheses. By synergizing LLMs, scientific explanation reconstruction, LLM-as-judge evaluation, and expert validation, the framework significantly outperforms direct hypothesis-generation baselines in data science. Notably, two high-scoring generated hypotheses were implemented as novel algorithms, both surpassing the original baseline models in performance and demonstrating strong cross-domain generalization potential.

Abstract

A scientific hypothesis is the first step in research and undergoes experimental validation, yet it also reflects a deep understanding of and reasoning about scientific phenomena. We introduce DN-Hypo-Pipeline, an AI-powered workflow based on large language models, designed to support structured scientific thinking and hypothesis generation by leveraging scientific explanations as prior knowledge. This pipeline assists researchers in deriving novel hypotheses from existing literature. Given the explanandum (i.e., the conclusion) of a research paper, it identifies underlying laws, theories, and principles, and reconstructs a new, yet-to-be-verified explanation for the observed phenomenon. We evaluated DN-Hypo-Pipeline in the field of data science modeling using three highly cited papers. Statistical inference, supported by both LLM-as-judge assessment and human expert evaluation, demonstrates that our pipeline is more effective than direct generation methods. Additionally, we validated the two highest-scoring generated hypotheses by developing corresponding novel algorithms, which outperformed the baseline models presented in the original papers. Beyond application in data science, DN-Hypo-Pipeline provides a theoretical framework that not only encompasses theory-guided data science modeling methods but also reveals a more fundamental structure of the modeling process. Moreover, this approach is essentially a generalization of theory-guided modeling, offering potential for extension to other domains and across a broader range of scientific disciplines.

Problem

• hypothesis generation
• scientific explanation
• large language models
• theory-guided modeling
• AI-driven workflow

Innovation

• hypothesis generation
• large language models
• scientific explanation
• theory-guided modeling
• AI-driven scientific discovery