fine-tune for compatibility

研究通过系统性调整学习率、批次大小等参数，对比LoRA和全微调方法，在不同模型和数据集上优化监督微调效果。

This work addresses the long-standing isolation among research domains such as alignment training, model organisms, and toy models, which has hindered empirical cross-pollination and led to redundant exploration and inefficiency. For the first time, it systematically transfers supervised fine-tuning (SFT) practices across these domains by integrating cross-model output training, mixed-strategy data, and benign fine-tuning to rigorously evaluate the portability of key findings. The study demonstrates three successful transfer effects: enhanced behavioral generalization, mitigation of capability degradation, and the critical insight that preserving capabilities alone is insufficient to ensure robustness in subsequent training phases. These results underscore both the efficacy and limitations of reusing methodologies across domains, thereby fostering more synergistic development across disparate research areas.

This work identifies a previously overlooked security alignment degradation risk in task-specific fine-tuning (e.g., multiple-choice fine-tuning): adversarial actors can manipulate dataset structure to induce harmful model outputs while preserving downstream task performance. To address this, we propose *Format- and Style-Consistent Safe Data Mixing*, a method that synthesizes safety-aligned data, mimics target task formatting, and aligns instruction styles to seamlessly inject safe samples into the fine-tuning pipeline. Our approach is the first to systematically demonstrate that task-level data structure can serve as an implicit attack surface and enables joint optimization of safety and task utility. Experiments across multiple benchmarks show over 50% reduction in harmful response rates while retaining ≥98% of original task accuracy—substantially outperforming existing defense baselines.

This work explores an efficient, lightweight paradigm for addressing software engineering tasks using only supervised fine-tuning (SFT), without relying on reinforcement learning or complex alignment techniques. To this end, we construct a high-quality hybrid dataset combining real-world and synthetically generated samples, and introduce several novel components: an error-masking mechanism, a software engineering–oriented curriculum learning strategy based on task difficulty, and a test-time scaling (TTS) approach integrated with trajectory validation. Our method achieves state-of-the-art performance among open-source models on SWE-bench Verified: SWE-Lego-Qwen3-8B and SWE-Lego-Qwen3-32B attain pass rates of 42.2% and 52.6%, respectively, which further improve to 49.6% and 58.8% under TTS@16.

A paradigmatic gap exists between supervised fine-tuning (SFT) and preference optimization (PO), hindering unified alignment. Method: We propose intuitive fine-tuning (IFT), a single-stage, single-strategy framework that unifies SFT and PO within an MDP formulation—modeling alignment as token-level preference estimation coupled with transition optimization. We theoretically show that SFT is a degenerate case of PO under zero preference signals, and introduce temporal residual connections to enable end-to-end alignment using only non-preference-labeled data at SFT-scale volume. Contribution/Results: Experiments demonstrate that IFT matches or surpasses the performance of standard two-stage SFT+PO across generation, reasoning, and fact-following tasks. Interpretable analysis on Frozen Lake further validates its policy efficacy. To our knowledge, this is the first work to achieve a unified optimization paradigm for SFT and PO.

This work addresses the critical challenge of dynamically determining when to perform continual fine-tuning of foundation models on resource-constrained devices under limited computational budgets to maximize performance. The problem is formally cast, for the first time, as a constrained Markov decision process, where the state encompasses model performance, remaining compute budget, and the relevance of incoming data to the historical distribution. The authors propose an online decision-making strategy based on an Actor-Critic reinforcement learning framework; when fine-tuning gains are predictable, dynamic programming is also integrated for optimal scheduling. Experimental results demonstrate that the proposed approach improves accuracy by over 4% compared to strong baselines under identical budgets and achieves 97% of the performance of full-parameter fine-tuning using only 25% of the fine-tuning steps.

研究探讨了客户服务LLM的多任务处理策略，通过多任务微调、顺序更新或模型合并的方法，发现多任务全微调在所有测试模型尺寸中表现最佳。

This work addresses the challenge of sparse and noisy observational data in few-shot, large-scale decision-making problems by introducing the pretraining–fine-tuning paradigm to this setting for the first time. The authors propose a problem-specific Transformer architecture that leverages domain knowledge to generate synthetic data for pretraining, followed by fine-tuning on a small amount of real-world data. Theoretically, they establish the first non-asymptotic generalization error bound, elucidating the synergistic mechanism between pretraining and fine-tuning and revealing a scaling law for fine-tuning. Empirically, high-capacity models effectively learn structural priors from synthetic data and adapt efficiently to real environments, with decision performance improving significantly as the instance scale grows.

Supervised fine-tuning (SFT) often compels models to fit observed tokens that may be noisy, non-unique, or inconsistent with prior knowledge, leading to suboptimal solutions. This work proposes the Q-target framework, which reframes SFT as a target distribution design problem by explicitly decoupling the strength of reliance on observed tokens from the strategy for assigning probabilities to alternative tokens. Building upon this insight, the authors introduce Target-SFT, a method that unifies SFT under the principled choice of a target distribution \( Q \), revealing more fundamental training principles and expanding the search space of objective functions. Extensive experiments across ten reasoning datasets and model configurations consistently demonstrate that Target-SFT significantly outperforms standard SFT, validating the effectiveness and generality of designing target distributions in supervised fine-tuning.

This study addresses the challenge that single models often fail to maximize accuracy in machine learning workflows, while existing systems lack adaptive switching capabilities. To overcome these limitations, this work proposes a dynamic data stream system based on multi-armed bandits. The core innovation lies in a novel bandit strategy that integrates runtime overhead with assertion probabilities, effectively mitigating the shortcomings of standard Thompson sampling and enabling intelligent model switching at runtime. Experimental results demonstrate that the proposed system improves accuracy by 23% over baseline methods and achieves a 48% efficiency gain compared to sequential execution.