π€ AI Summary
This work addresses catastrophic forgetting in continual learning by introducing a geometric perspective that frames forgetting as interference between tasks, quantified as an interference energy. Under frozen feature representations, the authors precisely measure this interference using the path-averaged curvature and theoretically link task support set separability to parameter retainability. Building on this insight, they propose a task-aware orthogonalization mechanism that enables structured parameter isolation and sharing, adaptively allocating parameters without requiring rehearsal or Fisher information. The method achieves lossless retention on structurally separable tasks and transforms irreversible forgetting into recoverable plasticity on non-separable tasks, consistently outperforming existing replay-free approaches across both similar and dissimilar task sequences.
π Abstract
Continual learning commonly relies on post-hoc mechanisms such as replay, elastic regularization, or distillation. This work argues that forgetting should instead be modeled directly as interference between tasks. In the frozen-feature regime, forgetting from learning a new task is exactly the interference energy induced on the old task. In deep networks, the same quantity is recovered through path-averaged curvature with minimal additional forward passes.
When task supports are disjoint, forgetting can be eliminated structurally and when task supports overlap in conflicting directions, a non-zero distortion floor is unavoidable. The same geometry optimally merges models through task-aware orthogonalization. From this analysis we derive Interference-Gated Functional Allocation (IGFA), a replay-free, Fisher-free method that shares directions when tasks align and protects them when they conflict. Across benchmarks, IGFA achieves lossless retention when tasks are structurally separable and moves unavoidable cost from irreversible forgetting into deferred but recoverable plasticity when they are not. It matches the strongest replay-free structural baselines on dissimilar-task streams and improves on unconditional projection when similarity makes transfer worth preserving.