large-scale screening

Critical materials synthesis knowledge is fragmented across vast volumes of unstructured scientific literature, severely impeding intelligent materials discovery. Method: We propose the first multimodal automated extraction framework for materials synthesis, integrating large language models (LLMs) and vision-language models (VLMs) with ontology-driven data modeling, LLM-as-a-judge quality assessment, and expert curation to jointly extract synthesis procedures, reaction conditions, and performance metrics from both text and figures with high accuracy. Contribution/Results: We construct LeMat-Synth (v1.0), a large-scale, structured dataset covering 35 synthesis methods and 16 material classes, comprising 25,000 high-quality synthesis protocols derived from 81,000 open-access papers. We also release a modular, open-source tool library to enable community-driven extension. This work establishes a scalable foundational infrastructure for modeling synthesis–structure–property relationships and enabling predictive materials design.

Existing citation recommendation systems struggle to identify essential references in AI/ML—such as experimental baselines, foundational methods, and core dependencies—leading to misjudgments of novelty and reproducibility issues. This work introduces the first focused study on “must-cite” scenarios, presenting MasterSet, a large-scale benchmark comprising over 150,000 papers from 15 top-tier conferences, with recommendations based solely on titles and abstracts. We propose a three-tier annotation framework integrating experimental baselines, core relevance, and in-text mention frequency, augmented by expert-validated large language model pipelines to ensure high-quality labels. Empirical results demonstrate that even strong baselines—including sparse retrieval, dense scientific embeddings, and graph-based methods—struggle on this task under Recall@K metrics, establishing MasterSet as a standardized evaluation platform and providing robust community baselines.

This work proposes a novel dataset discovery framework that leverages citation contexts from scientific papers to better capture the semantic intent behind research queries, addressing the limitations of existing dataset search engines that rely primarily on metadata and keyword matching and consequently suffer from low recall. By treating citation context as the core signal—combined with large-scale context extraction, large language model–guided pattern recognition, and provenance-preserving entity resolution—the approach significantly reduces dependence on incomplete or inconsistent metadata. Evaluated on eight computer science queries, the method achieves an average normalized recall of 47.47% (peaking at 81.82%), substantially outperforming Google Dataset Search and DataCite Commons. The framework’s novelty and practical utility have been affirmed by domain experts across multiple disciplines.

Existing scientific literature retrieval agents struggle to model complex inter-paper relationships—such as support, contradiction, or technological evolution—leading to fragmented knowledge and misinterpretation of research landscapes. This work introduces SciNet, the first systematically constructed benchmark dataset for relation-aware retrieval, encompassing 269 million papers and 8,940 tasks, and evaluates relational understanding at three granularities: ego-centric, pairwise, and path-wise. Built upon large-scale cross-disciplinary metadata and knowledge graphs, SciNet exposes a fundamental limitation of current mainstream retrieval methods, whose accuracy in modeling such relationships consistently falls below 20%. Incorporating this benchmark substantially improves the quality of downstream literature reviews by 25.3%, demonstrating the critical value of relation-awareness for scientific intelligence.

This study evaluates GPT-4’s capability to extract structured information on-demand from materials science literature, specifically assessing its zero-shot fidelity in reproducing two manually curated materials datasets. Methodologically, we employ an interdisciplinary, domain-expert–driven error attribution framework—integrated with rigorous human annotation—to systematically diagnose model output deviations across dimensions including numerical accuracy, contextual disambiguation, and unit standardization. Results reveal significant fidelity bottlenecks in GPT-4’s scientific information extraction (IE), particularly in precision-critical tasks. Our key contribution is the first application of deep expert-led error analysis to scientific IE evaluation, enabling quantitative characterization of large language models’ reliability boundaries in authentic research settings. We further propose a scalable, expert-validated benchmark for scientific IE assessment, advancing methodological foundations for high-fidelity AI-assisted scientific discovery.

Existing dataset discovery platforms rely on manual curation, resulting in limited coverage and delayed updates. To address this, this work proposes a “paper-first” automated paradigm and implements a lightweight system that continuously monitors arXiv to enable end-to-end, low-latency dataset extraction and dense semantic retrieval. The pipeline leverages a lightweight classifier, GROBID-based parsing, sentence-level description extraction, and a LaTeX source fallback mechanism. The system achieves an inference latency of only 11 ms and an F1 score of 0.94, improving dataset discovery efficiency by 80%. It has been deployed as a continuously updated online service, significantly accelerating researchers’ access to and retrieval of newly introduced datasets.

The exponential growth of scientific literature poses significant challenges in tracking the dynamic evolution of research fields. To address this, this work proposes an interactive and traceable system that, upon user query, retrieves relevant arXiv papers in real time and automatically generates semantic clusters and keyword labels based on titles and abstracts, visualizing thematic evolution over time. Departing from conventional manual categorization or domain-specific scripts, the system employs MiniLM embeddings, 10-dimensional UMAP for dimensionality reduction, and hierarchical clustering to enable end-to-end automated analysis. Evaluated across eight arXiv subject areas, 85% of the generated cluster labels were rated by users as semantically coherent, with domain experts particularly highlighting the system’s utility for rapid overview and exploratory analysis in fast-evolving technical domains.

This study addresses the challenges of fragmented literature and inefficient retrieval in aptamer research by proposing a localized, three-tier intelligent architecture that integrates a lightweight language model with deterministic algorithms to ensure both semantic comprehension and result reliability. The system employs a hierarchical response mechanism: it first attempts to automatically extract aptamer sequences; upon failure, it provides high-quality research leads—all without reliance on cloud services or subscription-based resources. Evaluated on 300 targets, the system successfully extracted sequences for 79% of them and delivered relevant literature or actionable clues for 84%, achieving a processing speed exceeding 900 targets per hour on a single machine. This approach significantly enhances the efficiency and accessibility of aptamer-related literature mining.

This study addresses the limitations of traditional manually curated biomedical databases, which suffer from high maintenance costs, delayed updates, and insufficient experimental context to capture nuanced data variations. To overcome these challenges, the authors propose a fully automated framework leveraging large language models (LLMs) to construct a large-scale, structured biomedical knowledge dataset enriched with fine-grained contextual information from full-text PubMed articles. The approach integrates LLM-driven ontology-aligned entity annotation, hybrid sparse-dense retrieval, and Starling—a multi-agent deep research system—to dynamically optimize knowledge extraction and retrieval strategies. The resulting resource comprises approximately 6.3 million records across six tasks, several of which constitute the largest publicly available datasets of their kind. Evaluation with state-of-the-art models demonstrates significantly lower error rates compared to prominent human-curated databases.

Current retrieval systems evaluate documents in isolation and rely on nDCG for aggregation, thereby overlooking inter-document relationships such as redundancy, conflict, and complementarity, which impedes accurate assessment of the overall quality of document sets. To address this limitation, this work proposes the first set-level evaluation–diagnosis–optimization closed-loop framework. It introduces SetwiseEvalKit, a three-tier, nine-dimension benchmark for comprehensive setwise evaluation, and Rubric4Setwise, a training-free method that translates multidimensional scoring rubrics into signals for set selection and ranking. The proposed approach achieves state-of-the-art downstream generation performance in both short- and long-text scenarios, attaining optimal results with fewer retrieved documents and retrieval rounds.