The Parts Are Greater Than the Sum: Automated Task Sequencing for Efficient Training of Multi-Policy LLMs

2026-07-31

AI Summary

This work addresses the limitations of conventional parameter-efficient fine-tuning (PEFT) methods when applied to sequences of heterogeneous tasks, where shared optimization paths often lead to task interference, negative transfer, and catastrophic forgetting. To mitigate these issues, the study introduces— for the first time—the concept of organizing distinct optimization paths within PEFT, proposing an automated multi-strategy QLoRA framework. This approach automatically groups and orders tasks to construct decoupled, independent adaptation pathways under a fixed parameter budget, enabling efficient co-training across diverse tasks without increasing model size. The method significantly enhances model generalization and transferability, achieving a state-of-the-art performance of 44.78 on the TRACE benchmark and substantially outperforming existing single-strategy PEFT approaches.

Abstract

Parameter-Efficient Fine-Tuning (PEFT) commonly adapts large language models using a single shared Low-Rank Adapter (LoRA). This shared optimization space often suffers from interference when adapting heterogeneous task sequences, leading to poor transfer and catastrophic forgetting. Existing approaches mainly improve adapter expressiveness by increasing parameter capacity or composing multiple adapters, yet they still rely on a shared optimization path. In this paper, we propose an optimization-path organization framework for parameter-efficient fine-tuning of large language models, implemented as an automatic multi-policy PEFT architecture. Specifically, optimization-compatible adaptation paths are automatically organized through task grouping and task sequencing under a fixed parameter budget. The organized optimization paths are implemented as independent Quantized Low-Rank Adapters (QLoRA), enabling heterogeneous tasks to be optimized in decoupled adaptation spaces while preserving positive transfer among compatible tasks. Experiments on the TRACE benchmark demonstrate that performance consistently improves from conventional single-policy PEFT to multi-policy PEFT, with the proposed automatic multi-policy framework achieving the best performance of 44.78 under the same trainable capacity. This suggests that optimization-path organization is more effective than simply increasing adapter capacity for heterogeneous parameter-efficient fine-tuning.

Problem

• Parameter-Efficient Fine-Tuning
• Task Interference
• Catastrophic Forgetting
• Heterogeneous Tasks
• Optimization Path

Innovation

• optimization-path organization
• multi-policy PEFT
• task sequencing
• Quantized Low-Rank Adapters
• parameter-efficient fine-tuning