Self-Evolving Deep Research via Joint Generation and Evaluation

2026-06-03

AI Summary

This work addresses the challenge in deep research report generation where the absence of explicit ground truth renders reward signals unverifiable and hinders optimization, particularly because static evaluators fail to adapt as solvers evolve. To overcome this limitation, the paper introduces SCORE, a novel framework that enables the first co-evolution of generation and evaluation modules under shared parameters. By employing an LLM-as-a-judge mechanism to construct a dynamic evaluation environment and incorporating meta-constraints to guide assessment dimensions and depth-aware search, SCORE breaks through the constraints of static evaluation. Extensive experiments on multiple deep research benchmarks demonstrate consistent improvements in report quality, validating the efficacy of jointly optimizing generation and evaluation.

Abstract

Large Language Models (LLMs) have become increasingly adopted in daily applications, with deep research standing out as a particularly important capability. Unlike traditional question-answering (QA) tasks, deep research report generation lacks definitive ground-truth, making reward design inherently unverifiable and limiting effective reinforcement learning. Existing approaches mitigate this challenge with LLM-as-a-judge and query-dependent evaluation rubrics, but they still rely on static evaluators that cannot adapt their standards as the solver improves, leading to insufficient and eventually saturated optimization pressure. We address this limitation with a \textbf{s}elf-evolving \textbf{co}-evolutionary training framework for deep \textbf{re}search evaluation and generation (SCORE), which tightly couples an evaluator and a solver in a shared-parameter learning process. Rather than treating generation and evaluation as isolated modules, we leverage their intrinsic connection to enable joint improvement within a single shared-parameter model. To restrict this process, we introduce a meta-harness, which dynamically controls the evaluation environment based on solver performance, encouraging valid evaluation dimensions and sufficiently deep evaluator search. Extensive experiments on deep research benchmarks demonstrate consistent improvement in report generation quality, showing that co-evolving evaluation and generation is a promising direction for training open-ended research agents.

Problem

• deep research
• reward design
• static evaluator
• reinforcement learning
• open-ended generation

Innovation

• self-evolving
• co-evolutionary training
• deep research
• shared-parameter model
• meta-harness