F$^{2}$DR: A Fine-Grained Full-Pipeline Reward Framework for DeepSearch Workflows

2026-09-17

AI Summary

为了解决现有奖励模型无法评估DeepSearch全流程复杂性的问题，提出了F2DR框架，从内容、轨迹和答案三个维度进行全面评估。

Abstract

With the widespread industrial deployment of Large Language Models (LLMs), DeepSearch has emerged as the dominant paradigm for resolving complex user queries. It typically operates through an iterative closed-loop workflow consisting of planning and reflection, information retrieval, and answer generation. However, existing reward models (RMs) and evaluation benchmarks are primarily designed for static single-turn tasks, failing to capture the full-pipeline complexity of DeepSearch workflows. To address this limitation, we propose F2DR, a fine-grained full-pipeline DeepSearch reward framework. F2DR evaluates DeepSearch workflows across three dimensions: Content, Trajectory, and Answer, enabling comprehensive process-level assessment. We further construct DeepSearch RM-Bench, a dedicated benchmark for evaluating RMs in DeepSearch scenarios. Extensive experiments demonstrate that F2DR achieves significantly higher evaluation consistency than self-evaluation-based baselines, while DeepSearch RM-Bench exhibits strong discriminative capability across existing open-source RMs. We will publicly release the complete DeepSearch RM-Bench dataset soon.

Problem

• DeepSearch
• Reward Models
• Evaluation Benchmarks

Innovation

• F2DR
• Full-Pipeline Reward Framework
• DeepSearch Workflows
• RM-Bench