Delayed Supervision for Test-Time Language Models

2026-09-26

AI Summary

This study addresses the vulnerability of test-time language models to overwriting previously learned facts due to the lack of explicit supervision over long-term memory. To mitigate this, we propose Late Supervision Training (LaCT), which introduces a long-interval delayed question-answering mechanism within simulated environments, integrating semantic supervision with standard next-token prediction to enhance the model's ability to distinguish between fact retention and revision. We post-train DeltaNet and RWKV-7 architectures on TextWorld trajectories and evaluate them on the BABILong and RULER benchmarks. Experimental results demonstrate that LaCT significantly improves memory persistence, yielding performance gains of 1.32 points for DeltaNet and up to 7.00 points for RWKV-7.

Abstract

Test-time language models adapt a compact memory while processing the input sequence. This perspective encompasses nonlinear fast-weight learning in LaCT, associative delta-rule updates in DeltaNet, and generalized delta-rule state updates in RWKV-7. Training these models to predict the next token does not explicitly require a fact to remain accessible after many subsequent memory updates. We study delayed supervision for this test-time memory: during post-training, ask a simulator-grounded question only after a long interval of unrelated events, and supervise its answer alongside ordinary next-token prediction. Questions are evaluated on disposable branches, so their answers never enter the continuing event stream. The construction distinguishes retention from revision: a retained fact must remain valid throughout the delay, whereas a revised fact must be answered with its latest value. We evaluate this approach on LaCT-760M and plain DeltaNet-1.3B using TextWorld training trajectories and shared BABILong and RULER evaluation panels, and include a separately reported RWKV-7 comparison. Relative to event-only training, delayed QA improves BABILong by 5.48 percentage points for LaCT and 1.32 points for DeltaNet, and single-needle RULER by 1.45 and 3.27 points, respectively. The RWKV-7 comparison reports gains of 4.60 and 7.00 points on its own panels. These results support delayed semantic supervision as a practical outer training objective for usable test-time memory, while leaving open how much of the benefit derives specifically from delay rather than general question-answering and answer-termination supervision.

Problem

• test-time language models
• delayed supervision
• memory retention
• compact memory
• next-token prediction

Innovation

• Delayed Supervision
• Test-Time Language Models
• Memory Retention
• Post-training
• Long-context Evaluation