ForecastCompass: Guiding Agentic Forecasting with Adaptive Factor Memory

2026-05-29

AI Summary

This work addresses the challenges of probabilistic forecasting in dynamic environments, where agents are often hindered by incomplete evidence, time pressure, and difficulties in calibration, while existing memory mechanisms fail to explicitly model reusable predictive factors and calibration knowledge. To overcome these limitations, the paper introduces ForecastCompass (FoCo), an adaptive factor-memory-based forecasting framework that establishes, for the first time, a hierarchical memory architecture tailored for prediction tasks. FoCo explicitly separates and maintains transferable predictive factors and calibration knowledge through a dual-channel memory system—comprising factor memory and reasoning memory—and employs a retrospection-driven, language-mediated memory revision mechanism to enable continual learning and iterative knowledge refinement. Evaluated on the Prophet Arena and FutureX benchmarks, FoCo significantly enhances both the accuracy and calibration of probabilistic forecasts generated by GPT-5-mini and Gemini-2.5-Flash models.

Abstract

Agentic forecasting is important for decision-making in dynamic environments, but it remains challenging because agents must reason from incomplete, time-limited evidence and produce calibrated probabilities before outcomes are resolved. Memory provides a natural mechanism for transferring experience from resolved forecasts to future prediction tasks. However, existing agent-memory methods are not tailored to forecasting, as they typically store past interactions, reflections, or factual associations without explicitly representing reusable predictive factors or calibration knowledge. We propose ForecastCompass (FoCo), an adaptive factor-based memory framework for agentic forecasting. FoCo organizes forecasting experience with a hierarchical forecasting-task taxonomy, enabling retrieval task-relevant forecasting knowledge. It maintains two complementary memory components: factor memory, which captures reusable predictive dimensions, and reasoning memory, which encodes probability updating, uncertainty handling, and calibration principles. Using retrospective analyses as learning signals, FoCo iteratively revises memory through a verbalized memory-revision procedure, enabling the agent to accumulate transferable forecasting knowledge over time. Experiments on Prophet Arena and FutureX with GPT-5-mini and Gemini-2.5-Flash show that FoCo improves both probabilistic accuracy and calibration.

Problem

• agentic forecasting
• memory
• predictive factors
• calibration
• dynamic environments

Innovation

• agentic forecasting
• adaptive factor memory
• forecasting calibration
• hierarchical task taxonomy
• memory-based reasoning