🤖 AI Summary
This work addresses the fundamental challenge that conventional neural networks cannot safely learn and infer concurrently, as parameter updates during inference often lead to unstable or even undefined outputs. To resolve this, the authors propose the DynamicGate MLP architecture, which decouples gating (routing) parameters from prediction (representation) parameters. For the first time, they rigorously establish—both structurally and mathematically—the sufficient conditions under which concurrent inference and learning are guaranteed to be safe. The approach ensures that valid model snapshots are maintained even under asynchronous or partial parameter updates, guaranteeing that every inference step yields a well-defined forward pass. This enables the deployment of stable online adaptive systems on edge devices, providing both theoretical assurance and practical foundations for continual learning.
📝 Abstract
Conventional neural networks strictly separate learning and inference because if parameters are updated during inference, outputs become unstable and even the inference function itself is not well defined [1, 2, 3]. This paper shows that DynamicGate MLP structurally permits learning inference concurrency [4, 5]. The key idea is to separate routing (gating) parameters from representation (prediction) parameters, so that the gate can be adapted online while inference stability is preserved, or weights can be selectively updated only within the inactive subspace [4, 5, 6, 7]. We mathematically formalize sufficient conditions for concurrency and show that even under asynchronous or partial updates, the inference output at each time step can always be interpreted as a forward computation of a valid model snapshot [8, 9, 10]. This suggests that DynamicGate MLP can serve as a practical foundation for online adaptive and on device learning systems [11, 12].