synthesize cross-disciplinary findings

This study addresses the limited depth and breadth of interdisciplinary knowledge integration caused by the poor reusability of underlying data in traditional literature reviews. We propose a "dual-track integration" conceptual framework that leverages the TIB Knowledge Loom to generate machine-readable outputs, combining systematic review methodologies with knowledge gap mapping techniques to comparatively evaluate manual extraction against automated approaches for knowledge synthesis. Our analysis reveals that only 8% of the examined literature provides reusable data, while demonstrating that constructing knowledge integration frameworks linking publications to research infrastructure effectively expands integration pathways. The core contribution of this work lies in identifying that ensuring the accessibility and executability of data, code, and workflows is essential for overcoming existing bottlenecks in knowledge integration.

This paper addresses reliability concerns and human-centered boundaries in the automated application of large language models (LLMs) for systematic literature reviews (SLRs) in Human-Computer Interaction (HCI). Method: Through empirical LLM experiments, human-AI collaborative workflow design, and HCI methodology analysis, it systematically delineates review stages amenable to automation (e.g., initial screening) versus those requiring human agency (e.g., thematic modeling, cross-study inference). Contribution/Results: The study proposes the “human-centered augmentation” ethical framework, rigorously defining LLMs’ capabilities and limitations in research synthesis. It delivers actionable, rigor-preserving guidelines for SLR practitioners and has been selected for a spotlight discussion at CHI ’25.

This study addresses the lack of executable and verifiable knowledge representations in existing meta-analyses, which hinders the traceability and reproducibility of critical analytical decisions. To overcome this limitation, the authors propose Executable Analytical Knowledge Representation (EAKR) and introduce MetaSynDec, an agent-based framework that, for the first time, enables explicit modeling, machine-actionable execution, and closed-loop validation of meta-analytic decisions. The system leverages large language models to generate structured knowledge and validates and executes it through deterministic, schema- and contract-based services. Evaluated across 58 synthesis units, EAKR successfully constructed all units, achieved exact evidence-set consistency in 75% of cases, and produced confidence intervals overlapping with published results in 98.2% of cases—substantially outperforming direct LLM-generated approaches.

One aspired outcome of empirical research on quantitative data is a variance theory, i.e., a quantification of the effect of an independent on a dependent variables. The validity of variance theories stems from the synthesis of multiple pieces of evidence, which increases its validity beyond the findings of a single study. However, research synthesis in SE is rare and if done mostly limited to purely narrative syntheses. At best, researchers perform meta-analyses to synthesize variance theories from several quantitative results. But even meta-analyses only produce reliable results when synthesizing exact replications yet fail to generalize from variations. We aim to extend the frontier of research synthesis beyond the state-of-the-art to systematically manage empirical evidence and its evolution. We apply method engineering to construct a framework for research synthesis from proven, individual method fragments. The framework allows researchers to put new evidence in a clear relation to an existing body of evidence and systematically expand knowledge about a studied phenomenon. We demonstrate the application of this framework to two fields of research by explicitly modeling the relationship between existing pieces of evidence. The framework puts three types of evolution of evidence into relation: (1) replications investigate the same hypothesis in a new context to improve external validity, (2) revisions challenge an existing hypothesis to improve internal validity, and (3) reanalyses replace analysis methods to improve conclusion validity. Through a systematic evolution of evidence and clear assessment criteria for each dimension of validity, the proposed framework can determine the frontier of a field of research. The framework provides a perspective to systematically evolve empirical evidence in SE, supporting more constructive and productive advances in our field.

Current evidence synthesis workflows adopt a flat, single-layer structure, leading to redundant method reconstruction across projects, conceptual drift, and unstable cross-project inference. To address these issues, we propose a three-tiered, inheritance-based meta-architecture comprising the *Methodological Law Layer*, the *Domain Abstraction Layer*, and the *Project Implementation Layer*. This design enforces hierarchical separation, explicit routing rules, and contamination control to ensure construct clarity and reproducibility. We introduce the first formal inheritance governance framework, defining three ontologically distinct entities—*Grandparent* (methodological laws), *Parent* (domain abstractions), and *Child* (project-specific implementations)—to enable systematic, lineage-aware methodological evolution and robust cross-project reasoning. Empirical evaluation demonstrates significant improvements in cross-project methodological consistency and replication rates. The architecture supports the construction of a scalable, multi-project evidence ecosystem and provides a sustainable, inheritable methodological infrastructure for long-term research programs.

Traditional retrosynthetic tools are constrained by reaction databases and struggle to devise creative synthetic routes for highly functionalized, polycyclic natural products. This work proposes SynthEx, a framework that leverages large language models to construct an intelligent agent system employing a strategy-first planning mechanism. By generating competitive synthetic strategies, integrating critical and routine steps, and incorporating self-reflection for iterative refinement, SynthEx achieves high-quality retrosynthetic planning. Notably, it produces key disconnections comparable to those devised by human experts—validated as authentic and feasible by chemists in blind evaluations. The method successfully designs highly convergent routes for over a thousand natural products and introduces SynthAtlas, an open-access database of these pathways, which has garnered recognition from domain experts.

This study addresses the challenges of synthesizing multi-source heterogeneous evidence—such as academic papers, reports, policies, and media content—which vary widely in quality and structure and entail high manual effort. The reliability of current large language models (LLMs) across individual synthesis subtasks remains unclear. To tackle this, the authors propose the Knowledge Synthesis Review (KSR) framework, decomposing the review process into four stages: screening, extraction, analysis, and synthesis. Using a high-agreement expert gold standard (92.2% agreement, κ=0.80), they conduct task-level evaluations of leading LLMs—including GPT-5, Claude Sonnet 4, and Gemini 2.5 Pro—and introduce a dynamic routing mechanism that automatically selects the best-performing model under human supervision. This model-agnostic, auditable, and transparent approach significantly enhances review efficiency and coverage. Experiments reveal no single model dominates all tasks: Claude Sonnet 4 achieves the highest screening accuracy (82.8%), while GPT-5 attains the best recall (91.8%). Moreover, multi-source synthesis uncovers critical themes—such as worker well-being, small and medium enterprises, and Global South perspectives—often missed in single-source analyses.

This study addresses the fragmentation of evaluation criteria for automated research systems and the difficulty of direct cross-task comparison. Employing a systematic literature review, it comprehensively examines evaluation designs across six task categories, including literature synthesis and ideation. By comparing benchmark construction and scoring protocols, this work proposes a complementary evaluation framework encompassing output-level, process-level, and human-subject assessments. It reveals the capability differences reflected by distinct designs and underscores the critical role of calibration specificity and resource budgets in performance interpretation. Furthermore, the project identifies gaps in diagnostic evaluation and provides recommendations for standardized reporting and auditing. Ultimately, these contributions offer practical guidance for benchmark selection and future research design in evaluating automated scientific discovery systems.

Existing datasets of scientific ideation trajectories struggle to comprehensively capture the full research process—from literature exploration and tool utilization to the evolution of intermediate artifacts and final proposals. This work proposes a reverse-to-forward synthesis mechanism that emulates the uncertainty, evidence integration, and phased convergence characteristic of real scientific inquiry through a Generator–Advisor architecture. By leveraging action–observation–editing sequence modeling, context-aware verification, and process-level supervision, the approach generates multi-turn trajectories aligned with authentic research practices, starting from high-quality papers and proposals. The study yields the first trajectory dataset spanning the complete scientific workflow and establishes a generalizable paradigm for synthesizing process-supervised data for scientific agents.

本文探讨了通过实用方法和无代码工具MUSE整合多源反馈，以支持计算设计，增强设计师的信心与灵活性。