TreeLoRA: Efficient Continual Learning via Layer-Wise LoRAs Guided by a Hierarchical Gradient-Similarity Tree

2025-06-12

AI Summary

To address the dual challenges of catastrophic forgetting and high computational overhead in continual learning of large pre-trained models (LPMs) on streaming data, this paper proposes a gradient-similarity-driven hierarchical LoRA adaptation framework. Our method dynamically activates task-aware LoRA modules per layer by constructing a gradient-similarity hierarchy based on K-d trees; efficiently explores task structure via a Lower Confidence Bound (LCB)-bandit algorithm; and optimizes parameter updates through sparse gradient propagation. Theoretical analysis guarantees convergence. Experiments on ViT and LLMs demonstrate that our approach achieves an average 5.7% improvement in task accuracy over state-of-the-art continual learning methods, reduces memory consumption by 42%, and accelerates training by 3.1×—striking a significant balance between performance and efficiency.

Technology Category

• Machine Learning: Life-Long and Continual Learning
• Computer Vision: Large Vision Models
• Natural Language Processing: (Large) Language Models

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Many real-world applications collect data in a streaming environment, where learning tasks are encountered sequentially. This necessitates continual learning (CL) to update models online, enabling adaptation to new tasks while preserving past knowledge to prevent catastrophic forgetting. Nowadays, with the flourish of large pre-trained models (LPMs), efficiency has become increasingly critical for CL, due to their substantial computational demands and growing parameter sizes. In this paper, we introduce TreeLoRA (K-D Tree of Low-Rank Adapters), a novel approach that constructs layer-wise adapters by leveraging hierarchical gradient similarity to enable efficient CL, particularly for LPMs. To reduce the computational burden of task similarity estimation, we employ bandit techniques to develop an algorithm based on lower confidence bounds to efficiently explore the task structure. Furthermore, we use sparse gradient updates to facilitate parameter optimization, making the approach better suited for LPMs. Theoretical analysis is provided to justify the rationale behind our approach, and experiments on both vision transformers (ViTs) and large language models (LLMs) demonstrate the effectiveness and efficiency of our approach across various domains, including vision and natural language processing tasks.

Problem

• Efficient continual learning for large pre-trained models
• Prevent catastrophic forgetting in sequential task learning
• Reduce computational burden in task similarity estimation

Innovation

• Layer-wise LoRAs guided by gradient-similarity tree
• Bandit techniques for efficient task similarity estimation
• Sparse gradient updates for parameter optimization