supervised fine-tuning

Conventional wisdom holds that fine-tuning is unsuitable for editing large language models (LLMs); however, this work reveals that its failure stems not from intrinsic methodological limitations, but from the prevailing single-sample, depth-first sequential editing paradigm—which induces over-optimization and cross-edit interference. Method: We propose LocFT-BF, a novel fine-tuning framework that replaces depth-first sequential updates with breadth-first (epoch-wise), mini-batch training, coupled with systematic localization of editable parameters to enable localized, stable, and efficient editing. Contribution/Results: LocFT-BF is the first approach to successfully adapt standard supervised training to model editing. It achieves state-of-the-art performance on large-scale editing tasks—up to 100K edits—and scales effectively to 72B-parameter models, significantly outperforming existing methods while preserving general model capabilities without degradation.

This paper addresses the fundamental challenge of balancing catastrophic forgetting and parameter efficiency when large pre-trained models continuously adapt to dynamic task streams. To this end, we propose the first unified theoretical framework for Parameter-Efficient Continual Fine-Tuning (PECFT). Our framework systematically organizes existing approaches along three dimensions: method taxonomy, evaluation metrics, and core challenges—integrating Parameter-Efficient Fine-Tuning (PEFT) techniques (e.g., adapters, LoRA, prompt tuning) with continual learning strategies (e.g., replay, regularization, architecture expansion). Through a comprehensive review of over 100 studies, we identify key trade-offs between performance and efficiency, and pinpoint scalable memory mechanisms and task-aware parameter updates as critical research frontiers. This work bridges a significant gap at the intersection of continual learning and PEFT, providing both theoretical foundations and practical guidelines for efficient, sustainable adaptation of large language models.

The conventional “fine-tune-then-compress” paradigm for lightweighting large language models (LLMs) during post-training incurs significant performance degradation and introduces redundant intermediate models. Method: This paper proposes the first end-to-end framework that jointly optimizes fine-tuning and structured compression—integrating progressive knowledge distillation, dynamic structured pruning, and low-rank parameter constraints directly into the downstream fine-tuning process to cooperatively shrink the parameter space. Contribution/Results: By eliminating the need to store and compute full-sized intermediate models, our approach reduces memory and computational overhead. On multiple benchmark tasks, it achieves an average accuracy gain of 2.1% at equivalent parameter counts and compresses model size by up to 4.3×, substantially mitigating performance decay inherent in conventional lightweighting pipelines.

To address structural distortion and topological inconsistency in high-dimensional parameter spaces (e.g., 4D tensors) induced by low-rank approximation in parameter-efficient fine-tuning, this paper proposes a structure-preserving low-rank core space modeling method. Unlike conventional low-rank adapters (e.g., LoRA), which are restricted to linear weight matrices, our approach explicitly models and preserves the intrinsic topological structure of the original high-dimensional parameter space—achieving compact and accurate reconstruction of N-dimensional parameter updates via high-order tensor decomposition. Evaluated across CV, NLP, and multimodal benchmarks, the method yields an average accuracy improvement of 1.8% under identical parameter budgets, while reducing structural distortion by 37%, significantly outperforming existing baselines.

Existing theoretical frameworks struggle to explain why larger-scale pre-trained models substantially reduce sample complexity on downstream tasks. This work proposes a novel theoretical framework—termed “caulking”—inspired by parameter-efficient fine-tuning methods such as adapters, low-rank adaptation, and partial fine-tuning. It establishes, for the first time, a provable relationship between the scale of pre-trained models and the sample complexity of downstream tasks. By rigorously linking stronger pre-training capabilities to reduced data requirements in transfer learning, this study not only addresses a critical gap in current theoretical understanding but also provides a solid foundation for empirically observed scaling laws, demonstrating that enhanced pre-training capacity can significantly decrease the number of samples needed for effective downstream adaptation.

This work addresses catastrophic forgetting in fine-tuning pretrained models, where newly acquired knowledge overwrites previously learned information. To mitigate this issue, the authors propose a function-preserving model expansion approach that mathematically duplicates and scales parameters of selected Transformer submodules during initialization. This technique enables stable training and faithful retention of original model capabilities without altering the initial functionality. By circumventing the traditional trade-off between plasticity and stability, the method achieves performance comparable to full fine-tuning while expanding only a minimal number of layers. Consequently, it fully preserves the model’s original knowledge and substantially reduces computational overhead.

This work proposes an alignment-aware fine-tuning framework that addresses the common oversight in existing methods—namely, their neglect of alignment objectives such as safety and hallucination mitigation—often exacerbating alignment deficiencies during task adaptation. The framework employs policy gradient regularization guided by external alignment signals to dynamically balance task performance and alignment constraints at the sample level. It further incorporates an adaptive gating mechanism to modulate gradients and enables the model to learn to abstain from responding to high-risk inputs. This approach intrinsically embeds conservative response behavior into the model without incurring additional inference overhead. Experimental results demonstrate that the framework significantly reduces harmful and hallucinatory outputs across general and domain-specific instruction-tuning benchmarks while preserving task performance, and exhibits strong robustness against adversarial fine-tuning and prompt-based attacks.

This work addresses the critical challenge of dynamically determining when to perform continual fine-tuning of foundation models on resource-constrained devices under limited computational budgets to maximize performance. The problem is formally cast, for the first time, as a constrained Markov decision process, where the state encompasses model performance, remaining compute budget, and the relevance of incoming data to the historical distribution. The authors propose an online decision-making strategy based on an Actor-Critic reinforcement learning framework; when fine-tuning gains are predictable, dynamic programming is also integrated for optimal scheduling. Experimental results demonstrate that the proposed approach improves accuracy by over 4% compared to strong baselines under identical budgets and achieves 97% of the performance of full-parameter fine-tuning using only 25% of the fine-tuning steps.

This work addresses the lack of systematic methodologies in model optimization, which often relies on heuristic choices and struggles to accommodate diverse deployment constraints. It formalizes model compression and acceleration as a constraint-aware multi-objective engineering decision problem, establishing a unified and actionable framework grounded in five key dimensions: data availability, latency, memory footprint, accuracy tolerance, and retraining budget. By integrating techniques such as quantization, pruning, knowledge distillation, parameter-efficient fine-tuning (PEFT), and inference optimization, the study proposes tailored optimization pipelines for four representative industrial scenarios, delivering a reproducible and quantifiable guide for technology selection.