FORT-Searcher: Synthesizing Shortcut-Resistant Search Tasks for Training Deep Search Agents

2026-06-10

AI Summary

This work addresses the susceptibility of existing deep-search agents to shortcut effects, wherein they rely on superficial cues rather than genuine reasoning. The authors propose FORT, a novel framework that formalizes shortcut-aware difficulty for the first time and identifies four actionable categories of shortcut risks. FORT synthesizes shortcut-resistant training data through four stages: entity selection, evidence graph construction, question formulation, and adversarial optimization. By integrating evidence graph construction, adversarial refinement, supervised fine-tuning, and trajectory analysis—including metrics such as answer hit time and prior shortcut rate—the generated data significantly prolongs effective search trajectories and suppresses shortcut behaviors. A search agent trained on this data, FORT-Searcher, achieves state-of-the-art performance among comparable models on open-source deep-search benchmarks.

Abstract

Training deep search agents requires verifiable questions whose answers remain unavailable until sufficient evidence has been acquired through search. Existing synthesis methods often increase apparent difficulty by enriching graph structures, but structural complexity alone does not guarantee realized search difficulty: the intended search process can collapse through a cheaper identifying route. We formalize this gap with a shortcut-aware difficulty framework and identify four actionable shortcut risks: evidence co-coverage, single-clue selectivity, exposed constants, and prior-knowledge binding. To diagnose their realized effects, we use trajectory signatures including solving cost, answer hit time, and prior-shortcut rate. Guided by this framework, we introduce FORT, a Framework of Shortcut-Resistant Training-Data Synthesis. FORT constructs shortcut-resistant training data by controlling shortcut risks across entity selection, evidence graph construction, question formulation, and adversarial refinement. Experiments show that FORT induces longer pre-answer search and fewer shortcut patterns than existing open-source deep search datasets. Using the resulting trajectories, we train FORT-Searcher with supervised fine-tuning (SFT) only, and it achieves the best overall performance among comparable-size open-source search agents on challenging deep search benchmarks. Relevant resources will be made available at https://github.com/RUCAIBox/FORT-Searcher.

Problem

• shortcut-resistant
• deep search agents
• search task synthesis
• evidence graph
• training data

Innovation

• shortcut-resistant
• deep search agents
• training data synthesis
• evidence graph
• trajectory signatures