Is More Privileged Information Better? From Solution Traces to Problem-Solving Structure in Self-Distilled Reasoning

2026-08-02

AI Summary

This work addresses the limited generalization of traditional on-policy self-distillation methods, which rely on full solutions unavailable during inference as supervision signals. To overcome this, the authors propose Problem-Space Guided On-Policy Self-Distillation (PS-OPSD), which reformulates privileged information into structured guidance comprising only the initial state, goal conditions, constraints, and state-transition paths. This enables the student model to reason solely from the problem input. PS-OPSD integrates a trajectory anchoring mechanism within an on-policy distillation framework, effectively decoupling training from inference. Evaluated across three mathematical reasoning benchmarks, PS-OPSD achieves state-of-the-art question-only accuracy across model scales ranging from 1.7B to 8B parameters, demonstrating the critical importance of guidance relevance and path coherence.

Abstract

On-policy self-distillation (OPSD) improves reasoning by using a privileged view of a model conditioned on reference solutions to supervise a student view that observes only the question. However, the teacher-provided token-level targets may depend on reference-specific information unavailable at inference time. We propose Problem-Space-Guided OPSD (PS-OPSD), which replaces the complete solution with trajectory-grounded guidance describing the initial state, goal conditions, constraints, and a selected state-transition path. The student rollout and OPSD objective remain unchanged. Across three mathematical reasoning benchmarks and model scales ranging from 1.7B to 8B, PS-OPSD achieves the highest aggregate question-only accuracy among the compared methods. Controlled experiments further indicate that guidance relevance and path coherence contribute to these gains, highlighting the representation of privileged information as an important design choice in OPSD.

Problem

• privileged information
• self-distillation
• reasoning
• solution traces
• problem-solving structure

Innovation

• Problem-Space Guidance
• On-Policy Self-Distillation
• Privileged Information
• Trajectory-Grounded Reasoning
• Mathematical Reasoning