Black-Mamba: Biologically-Inspired Leaky Accumulation for Conceptual Knowledge under Distribution Drift

2026-07-21

AI Summary

Real-world forecasting is often hindered by distribution shifts, and existing methods frequently misinterpret transient noise as persistent changes, leading to unnecessary model updates. This work proposes an online adaptation mechanism gated by cumulative surprise evidence, incorporating a biologically inspired leaky-integrator dynamic memory module that triggers event-driven state updates only upon detecting significant and sustained distribution shifts. By decoupling adaptation from transient prediction errors, the approach integrates test-time adaptive sequence modeling with neuroscience-inspired update criteria. Evaluated across multiple non-stationary forecasting benchmarks, the method achieves state-of-the-art performance while substantially reducing the number of memory updates during inference, thereby enhancing both adaptation efficiency and robustness.

Abstract

Forecasting under real-world conditions is inherently non-stationary, as the conditional distribution of future observations evolves over time. Recent test-time adaptive sequence models address this challenge by updating internal states during inference, but tie adaptation to instantaneous prediction errors or surprise. This coupling can conflate persistent distribution shift with stochastic innovations, leading to unnecessary updates and inefficient adaptation. We introduce Black-Mamba, a test-time adaptive forecasting architecture that formulates online adaptation as evidence-gated state tracking under distribution drift. The model augments a base predictor with a dynamic memory updated when temporally accumulated surprisal provides sufficient evidence of a regime change. This turns adaptation into a selective, event-driven process rather than a continuous one. Across multiple forecasting benchmarks with non-stationary dynamics, Black-Mamba achieves competitive or improved predictive performance compared to existing test-time adaptation methods while significantly reducing the number of memory updates during inference. Together with mathematical analysis and biological evidence, these results suggest that accumulated surprisal provides a principled signal for distinguishing persistent drift from transient noise, yielding more efficient and robust adaptation.

Problem

• distribution drift
• test-time adaptation
• non-stationary forecasting
• surprisal accumulation
• conceptual knowledge

Innovation

• test-time adaptation
• distribution drift
• accumulated surprisal
• event-driven adaptation
• non-stationary forecasting