Learning from Synthetic Data without Model Collapse in Iterative Instruction Tuning

2026-07-18

AI Summary

This work addresses the issue of "model collapse" in iterative instruction tuning, where large language models trained on synthetic data often exhibit capability polarization—over-strengthening their strong skills while continuously degrading weaker ones. To mitigate this, the authors propose KITE, a novel framework that integrates failure-guided data generation with a boundary-aware uncertainty filtering mechanism to strategically target and refine the model’s knowledge boundaries. Extensive experiments demonstrate that KITE consistently outperforms strong baselines across multiple open-source large language models and benchmark datasets, effectively curbing capability polarization and yielding more stable, balanced performance improvements.

Abstract

Model collapse is a central challenge in learning from synthetic data: as later-generation large language models (LLMs) are trained on an increasing proportion of model-generated data, performance can degrade due to narrowed coverage and accumulated bias. Existing work mainly studies how to bound this degradation. In iterative model evolution, however, the more meaningful objective is to ensure that each successive model improves over its predecessor, which requires diagnosing collapse at a granularity that is actionable for data curation. We study this problem in synthetic data self-improving for instruction tuning. We show that collapse in this setting is not simply uniform performance degradation, but can appear as a polarization of competence, where synthetic training reinforces already strong skills while further degrading weak ones. Motivated by this observation, we propose KITE (Knowledge-boundary Instruction Tuning via Exploration), a two-stage framework that combines failure-guided data generation with boundary-aware uncertainty curation. Experiments across several datasets and multiple open-source LLMs show that KITE yields more stable improvement than strong synthetic-data baselines.

Problem

• model collapse
• synthetic data
• instruction tuning
• iterative model evolution
• capability polarization

Innovation

• model collapse
• synthetic data
• instruction tuning
• KITE
• uncertainty curation