What to Preserve in Recursive Computation: A Local Predictive Sufficiency Principle

2026-10-03

AI Summary

This study addresses the loss of long-horizon predictions and insufficient information retention caused by compression in recursive computation. To this end, it proposes the principles of local predictive sufficiency and recursive predictive closure. Methodologically, an empirical predictive deficiency metric is derived via variational features, and half-space constraints are constructed using predictive sensitivity to projectively correct optimizer updates, thereby preserving predictive information. This framework is the first to link local sufficiency with recursive closure, establishing a task-agnostic, universal training principle applicable without modifying backbone architectures. Evaluated across temporal graphs, language memory, and vision-language-action control tasks, the approach matches or improves performance under identical compression budgets, yielding particularly significant gains in scenarios demanding extensive recursion.

Abstract

Recursive computation repeatedly compresses or reuses intermediate states, creating a simple tension: information that must remain useful across longer recursive paths is also exposed to more opportunities for loss before reaching the final prediction. Existing reconstruction or local-prediction objectives provide tractable supervision, but do not ensure that the retained information remains sufficient for subsequent recursive computation. We identify local predictive sufficiency with recursive predictive closure: controlling local predictive deficiencies at individual interfaces controls the resulting discrepancy at the root. We then turn this principle into a tractable training procedure. Starting from a variational characterization, we derive finite predictive tests and an empirical predictive deficiency that measures predictive value retained across compression. Its predictive sensitivities define margin-relaxed half-space constraints on parameter updates, and we project the host optimizer's proposed update onto their intersection only when predictive preservation would otherwise be violated. Across temporal graphs, language memory, vision-language-action control, and recursive self-improvement, the method matches or improves the corresponding host models under matched compression budgets, with larger gains under heavier recursive or memory demands, while better preserving predictive information across successive transformations. Crucially, the same task-agnostic predictive-preservation principle is instantiated across all four settings through host-compatible interventions while keeping the endpoint task, backbone, and evaluation protocol fixed. These results establish predictive preservation at recursive interfaces as a general training principle for recursive compression.

Problem

• recursive computation
• predictive sufficiency
• information compression
• intermediate states
• predictive preservation

Innovation

• Local Predictive Sufficiency
• Recursive Computation
• Predictive Preservation
• Variational Characterization
• Margin-Relaxed Constraints