Does Every User Need a Private LoRA? Decoupling Personalization from Per-User Adaptation

2026-10-01

AI Summary

Existing personalized large language models suffer from limited scalability by storing complete adaptation parameters for each user. This work proposes LINEUP, a framework that decouples personalization capacity from user-specific parameters to enable efficient scaling. Its core innovation lies in constructing a shared low-rank adapter library integrated with user-conditioned retrieval and query-dependent calibration mechanisms, compressing per-user trainable parameters from millions to merely eight scalars while providing theoretical guarantees on finite-history risk. Experimental results demonstrate that the proposed method achieves comprehensive superiority across twelve metrics spanning six tasks, notably reducing RMSE by 11.4% on LaMP-3. These findings validate the feasibility of leveraging shared parameter architectures to support large-scale personalized modeling.

Abstract

Personalized large language models often require a complete adaptation state for each user. However, this paradigm scales poorly as the user population grows. We revisit this design through the lens of personalization capacity allocation: how much adaptation capacity can be shared across users, how the shared capacity should be composed, and how much must remain user-specific. We answer them through three complementary empirical analyses. We find that independent user adapters contain substantial cross-user reusable structure, that the utility of reusable directions reflects both user relevance and variation across queries, and that user histories provide transferable signals for compact individual correction. Motivated by these findings, we propose LINEUP. It learns a bank of reusable low-rank personalization factors, composes them through user-conditioned recall and query-dependent calibration, and restricts target-user adaptation to a tiny user code over a shared correction space. This design decouples expressive personalization capacity from per-user trainable state. Each target user optimizes only eight scalars, while all shared components remain fixed. By comparison, the evaluated private-LoRA configuration uses 4.19 million per-user parameters. Our theoretical analysis gives a finite-step, finite-history risk bound and sufficient conditions for user-code refinement to improve on history initialization. Across six tasks spanning personalized classification, prediction, and generation, LINEUP leads on all 12 metrics, each averaged over three independent runs (e.g., reducing LaMP-3 RMSE by 11.4% relative to the strongest baseline). It maintains advantages under limited history. These results show that rich personalization can be supported primarily by reusable, conditionally composed shared capacity, while independent user adaptation remains confined to a tiny correction state.

Problem

• Personalized LLMs
• Scalability
• Parameter efficiency
• Per-user adaptation
• Low-rank adaptation

Innovation

• Personalized LLMs
• Low-Rank Adaptation
• Parameter Sharing
• User-Conditioned Composition
• Decoupled Personalization