SDO: Structure-Aware Data Organization for Efficient LLM Post-Training

2026-07-29

AI Summary

Current post-training approaches for large language models rely on static data organization, which fails to dynamically adapt to evolving sample exposure during optimization, often resulting in redundant updates alongside under-optimized instances. To address this, this work proposes the Structure-driven Dynamic Organization (SDO) framework, which employs a representation-space-structure-guided mechanism to construct locally consistent mini-batches within each epoch by leveraging KNN neighborhoods derived from frozen external embeddings. Additionally, SDO incorporates per-sample exposure histories across epochs to adjust sampling probabilities, enabling efficient and balanced training without requiring model warm-up. Experimental results demonstrate that SDO significantly accelerates convergence—particularly in early-to-mid training stages—and enhances gradient coherence as well as the balance of accuracy across diverse question types in SFT, DPO, and GRPO tasks.

Abstract

Post-training of large language models is expensive, and existing efficiency improvements mainly focus on selecting informative samples or designing training schedules. However, data organization itself is usually treated as a static preprocessing step: embedding-based grouping methods construct fixed partitions before training and cannot adapt to the evolving sample exposure during optimization. As a result, all samples receive similar exposure despite their different optimization needs, leading to redundant updates for some samples while leaving others under-optimized. To address this problem, we propose SDO (Structure-Aware Data Organization), a plug-and-play data organization framework with an exposure-driven feedback mechanism that organizes mini-batch composition and sample exposure according to representation-space structure. SDO operates epoch by epoch on frozen external embeddings, avoiding model warm-up training overhead: within each epoch, locality-aware batching forms coherent mini-batches via KNN neighborhood traversal; across epochs, exposure-balanced scheduling records per-sample participation and reduces the sampling probability of over-exposed samples to preserve long-term coverage. Across SFT, DPO, and GRPO, SDO accelerates convergence, with the largest gains observed in the early-to-mid phase, producing more coherent gradients and more balanced accuracy across question types without permanently excluding training samples.

Problem

• data organization
• post-training
• sample exposure
• large language models
• training efficiency

Innovation

• Structure-Aware Data Organization
• exposure-driven feedback
• locality-aware batching
• exposure-balanced scheduling
• embedding-based grouping