Distilling Temporal Search and Reasoning: Evolving LLMs for Future Prediction via Harness-Assisted Efficient Data Synthesis

2026-07-28

AI Summary

This work addresses key limitations in existing future event prediction methods—namely, inefficient synthetic data generation, severe temporal leakage, and excessive reliance on external tools, which collectively degrade model performance when such aids are removed. To overcome these issues, the authors propose a temporal truncation mechanism that enables efficient sampling from historical events while preventing temporal leakage, thereby enhancing the generation of high-quality temporal reasoning data for distilling parametric language models. This approach substantially reduces dependence on rejection sampling and unresolved queries, naturally broadening the model’s temporal reasoning scope. Integrated within a Tool-Integrated Reasoning (TIR) framework and supported by large-scale corpus construction and process-oriented evaluation, the resulting student model achieves state-of-the-art performance on temporal reasoning and prediction tasks, demonstrating that synthetically generated data can be effectively internalized as predictive capability.

Abstract

Future event prediction carries broad social impact yet remains challenging. SOTA approaches augment LLMs with external agent frameworks whose predictive capability vanishes once the harness is removed. While recent Tool-Integrated Reasoning (TIR) internalizes deep search for multi-hop retrieval of facts, forecasting further demands temporal search and reasoning over historical trends and dynamic shifts. The key obstacle is data: historical queries induce temporal leakage that degrades forecasting into retrieval. Prior works either freeze information gathering with static observations, or rely on rejection sampling or unresolved fresh queries that discard vast amounts of data, degrading synthesis efficiency. We propose a time-truncation harness that enforces a temporal cut-off at every turn, enabling TIR-style sampling from historical events, reducing temporal leakage and reliance of rejection sampling or unsolved queries, increasing the sampling efficiency. We further build a large-scale corpus and a process-based metric and show that our harness naturally induces a broader temporal breadth of search and raises the proportion of high-quality data, further increasing the efficiency and reducing the reliance on complex rubrics. Distillation experiments show that students trained on harness-intervened data achieve the best performance, demonstrating harness-assisted model evolving that turns higher quality temporal search and reasoning data into a parametric advancement of the students.

Problem

• future prediction
• temporal reasoning
• data synthesis
• temporal leakage
• LLMs

Innovation

• time-truncation harness
• temporal reasoning
• data synthesis
• future prediction
• distillation