iSDFT: Information-Proximal Self-Distillation for Continual Learning in LLMs

2026-09-21

AI Summary

为解决持续学习中信息转移控制问题，提出iSDFT方法，通过在每个预测状态选择满足教师信息约束的最近分布，提高模型在多种任务上的表现。

Abstract

On-policy self-distillation fine-tuning (SDFT) learns new skills from demonstrations while reducing forgetting, but it always distils toward the full demonstration-conditioned teacher. This fixes teacher influence at the full-teacher endpoint, providing no control over how much demonstration information should be transferred at each prediction state. We introduce Information-Proximal SDFT (iSDFT), which instead treats the teacher as a budgeted source of information. At each token, iSDFT selects the distribution closest to the current student that satisfies a prescribed teacher-information constraint, yielding a closed-form exponential target with a locally determined tilt. To control cumulative drift, we further anchor the student to its frozen base policy. Across four heterogeneous LLM backbones and two specialisation tasks, iSDFT improves vanilla SDFT in 7 of 8 model-task settings and matches it in the remaining one. It also provides tighter retention on the original SDFT benchmark suite, with 73% of evaluations remaining within 0.5 points of the base model versus 52% for the strongest baseline, while achieving the largest mean improvement on all ten additional mathematics, coding, and competition-mathematics benchmarks. These results show that controlling how much and when teacher information is introduced improves specialisation while preserving broader capability.

Problem

• self-distillation
• continual learning
• demonstration information
• teacher influence
• information control

Innovation

• Information-Proximal SDFT
• teacher-information constraint
• cumulative drift control
• closed-form exponential target
• specialisation improvement