Transforming Science with Large Language Models: A Survey on AI-assisted Scientific Discovery, Experimentation, Content Generation, and Evaluation

2025-02-07

AI Summary

This study addresses the challenge of integrating large multimodal language models (LMMs) across the entire scientific research lifecycle—from intelligent literature retrieval and hypothesis generation to experimental design, multimodal content synthesis (text, figures, tables), and AI-augmented peer review. We propose a comprehensive “AI4Science” paradigm, unifying knowledge graph–enhanced retrieval, multimodal understanding and generation, automated evaluation metrics, and trustworthiness analysis. Synthesizing over 100 state-of-the-art works, we systematically survey task-specific SOTA methods, benchmark datasets, and empirical performance, while identifying critical limitations. Crucially, we introduce an ethics-aware governance framework and scientific integrity risk assessment mechanism, establishing foundational evaluation benchmarks for trustworthy AI-assisted research. Our contribution culminates in a human-AI collaborative, scalable research paradigm grounded in technical rigor and responsible innovation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Philosophy and Ethics of AI: Safety, Robustness & Trustworthiness
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Large language models for search
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

With the advent of large multimodal language models, science is now at a threshold of an AI-based technological transformation. Recently, a plethora of new AI models and tools has been proposed, promising to empower researchers and academics worldwide to conduct their research more effectively and efficiently. This includes all aspects of the research cycle, especially (1) searching for relevant literature; (2) generating research ideas and conducting experimentation; generating (3) text-based and (4) multimodal content (e.g., scientific figures and diagrams); and (5) AI-based automatic peer review. In this survey, we provide an in-depth overview over these exciting recent developments, which promise to fundamentally alter the scientific research process for good. Our survey covers the five aspects outlined above, indicating relevant datasets, methods and results (including evaluation) as well as limitations and scope for future research. Ethical concerns regarding shortcomings of these tools and potential for misuse (fake science, plagiarism, harms to research integrity) take a particularly prominent place in our discussion. We hope that our survey will not only become a reference guide for newcomers to the field but also a catalyst for new AI-based initiatives in the area of"AI4Science".

Problem

• AI enhances literature search efficiency
• AI aids in generating scientific content
• AI automates peer review processes

Innovation

• Large multimodal language models
• AI-assisted scientific discovery
• Automatic peer review