fine-tuning strategy comparison

This work investigates whether pretraining metrics—such as perplexity—reliably predict downstream performance of large language models (LLMs) after fine-tuning, aiming to improve model selection efficiency under fixed computational budgets. The authors formulate checkpoint selection as a pairwise classification task and systematically evaluate 50 distinct 1B-parameter LLM variants across diverse downstream tasks, revealing that perplexity is frequently misleading. They propose novel unsupervised and supervised proxy metrics, which reduce prediction error rates by over 50% in multi-task supervised fine-tuning (SFT) evaluation. This study is the first to empirically demonstrate a non-monotonic relationship between pretraining metrics and downstream performance. The proposed proxies exhibit strong cross-task generalization and practical utility, offering a trustworthy, task-aware evaluation paradigm for optimizing pretraining strategies toward downstream objectives.

研究通过系统性调整学习率、批次大小等参数，对比LoRA和全微调方法，在不同模型和数据集上优化监督微调效果。

This study addresses the high cost and sensitivity of large language model fine-tuning to data quality and hyperparameters, highlighting the need for pre-training performance prediction. The authors propose TuneAhead, a novel framework that enables accurate forecasting of fine-tuning outcomes by constructing a lightweight regression model from static data descriptors and dynamic features derived from short, standardized probing runs. Integrated SHAP analysis provides interpretable diagnostics to support informed “proceed/abort” decisions prior to full-scale training. Evaluated across 370 hold-out tests, TuneAhead achieves an RMSE of 1.47 percentage points, with 95.1% of predictions falling within ±3 percentage points of ground truth—significantly outperforming baseline methods such as Early-Stop Extrapolation and ProxyLM.

In large-scale pretraining, learning rate scheduling critically influences both training efficiency and model performance. This work proposes two paradigms—Fitting and Transfer. The Fitting paradigm establishes, for the first time, a scaling law for learning rate search factors, reducing hyperparameter tuning complexity from O(n³) to O(n·C_D·C_η). The Transfer paradigm extends μTransfer to Mixture-of-Experts (MoE) architectures and generalizes it across multiple hyperparameter dimensions, including depth, weight decay, and token length. Empirical results demonstrate that while μTransfer exhibits limited scalability in large-scale settings, the Fitting paradigm—grounded in the derived scaling law—offers superior scalability and practicality, providing a systematic guideline for hyperparameter tuning in industrial-scale pretraining.

This study investigates learning dynamics in large language model (LLM) fine-tuning, focusing on three critical phenomena: (1) cross-task factual transfer and response repetition hallucinations in instruction tuning; (2) the “squeezing effect”—an anomalous decline in target response probability—under prolonged preference optimization; and (3) the superiority of on-policy over off-policy direct preference optimization (DPO). We propose an influence-function-based stepwise attribution method and construct a response-level influence-path model, providing the first unified explanation for these phenomena. We formally identify and characterize the squeezing effect as arising from excessive contraction of the policy distribution, thereby revealing the intrinsic advantage of on-policy training. Leveraging this insight, we design a novel alignment-optimization strategy. Experiments demonstrate that our framework significantly improves fine-tuning stability and alignment performance, offering both theoretical foundations and practical solutions for efficient, controllable LLM fine-tuning.

This study challenges the common assumption that models exhibiting similar performance after supervised fine-tuning (SFT) are functionally equivalent, by demonstrating that the data used in the final stage of pretraining critically influences subsequent alignment behavior. Through controlled experiments—where only the last 500 million tokens of pretraining data are varied while keeping SFT and post-training procedures identical—the authors show that ending pretraining with safety-oriented text significantly preserves a model’s ability to refuse harmful requests, an effect absent with other data types. This finding is replicated across another model family, revealing for the first time that late-stage pretraining data selectively shapes how models evolve during preference optimization and reinforcement learning. The results question evaluation practices that rely solely on post-SFT performance as a proxy for alignment capability.

研究探讨了客户服务LLM的多任务处理策略，通过多任务微调、顺序更新或模型合并的方法，发现多任务全微调在所有测试模型尺寸中表现最佳。

This study addresses the high fine-tuning costs and associated risks—such as knowledge degradation, diminished instruction-following capability, and increased hallucination—faced by small-scale large language models in cybersecurity question-answering tasks. To mitigate these issues, the authors propose the FiT diagnostic framework, which evaluates model suitability prior to fine-tuning along three dimensions: lexical recognition, parametric knowledge, and contextualization of retrieved information. The work establishes the first task-oriented diagnostic system tailored for cybersecurity QA, integrating knowledge-focused and instruction-focused fine-tuning paradigms with retrieval-augmented evaluation. Through empirical analysis of five open-source 7B models, the study reveals that fine-tuning generally impairs lexical and parametric knowledge: knowledge-focused fine-tuning induces mild yet consistently ranked degradation, whereas instruction-focused fine-tuning triggers severe knowledge collapse while preserving the ability to contextualize retrieved information. Notably, FiT scores effectively predict post-fine-tuning performance trends.

This work addresses the high cost of large model fine-tuning by tackling the challenge of accurately predicting post-fine-tuning performance beforehand—a task whose theoretical limits remain unclear. We formulate pre-fine-tuning performance prediction as a stochastic estimation problem under information constraints and introduce a predictive risk decomposition framework that separates it into an irreducible intrinsic limit and an optimizable variance term, thereby revealing fundamental bounds on predictability. Leveraging information theory and statistical learning theory, we establish a theoretical lower bound on variance decay through optimization and construct a predictability phase diagram that delineates three distinct task regimes. Experiments on both synthetic and real-world benchmarks validate the efficacy of this phase diagram, and our proposed budget-optimal probing strategy significantly enhances prediction efficiency, offering both theoretical grounding and practical tools for pre-fine-tuning decision-making.

This work addresses the semantic gap between evaluation metrics and training data in large model pretraining, which hinders precise diagnosis and remediation of capability deficiencies. The authors propose “capability slices” as fundamental units aligning evaluation and data, establishing a bidirectional classification framework that links evaluation tasks with non-instructional training data through explicit mapping rules. This enables a closed-loop pipeline from evaluation failures to targeted data interventions. For the first time, the approach supports auditable and systematic reasoning that translates evaluation signals into data corrections, moving beyond intuition-driven tuning paradigms. Experiments demonstrate its efficacy in both directions: repairing specific training loss components restores BBH performance to 66.44, while targeted data sampling boosts AIME2025/2026 Pass@128 from 6.67/0.00 to 26.67.