Democratizing Large Language Models via Personalized Parameter-Efficient Fine-tuning

2024-02-06

Conference on Empirical Methods in Natural Language Processing

AI Summary

To address core challenges in large language model (LLM) personalization—including lack of model ownership, limited customization, high privacy risks, and difficulty modeling dynamic user behavior—this paper proposes the “One PEFT Per User” (OPPU) framework. OPPU assigns each user a dedicated, lightweight parameter-efficient fine-tuning (PEFT) module (e.g., LoRA or Adapter), ensuring private parameter ownership and enabling localized, on-device updates. It innovatively integrates parametric preference modeling with non-parametric retrieval augmentation to support adaptation to behavioral drift and robust modeling across varying user activity levels. Evaluated on seven LaMP benchmark tasks, OPPU significantly outperforms prompt-engineering baselines. It demonstrates robustness to changes in user history formatting, compatibility with diverse PEFT techniques, and strong generalizability—validating the feasibility, practicality, and privacy-preserving nature of user-granular personalized fine-tuning.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: (Large) Language Models
• Search and Optimization: Learning to Search

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems

Abstract

Personalization in large language models (LLMs) is increasingly important, aiming to align the LLMs’ interactions, content, and recommendations with individual user preferences. Recent advances have highlighted effective prompt design by enriching user queries with non-parametric knowledge through behavior history retrieval and textual profiles. However, these methods faced limitations due to a lack of model ownership, resulting in constrained customization and privacy issues, and often failed to capture complex, dynamic user behavior patterns. To address these shortcomings, we introduce One PEFT Per User (OPPU), employing personalized parameter-efficient fine-tuning (PEFT) modules to store user-specific behavior patterns and preferences. By plugging in personal PEFT parameters, users can own and use their LLMs individually. OPPU integrates parametric user knowledge in the personal PEFT parameters with non-parametric knowledge from retrieval and profiles, adapting LLMs to user behavior shifts. Experimental results demonstrate that OPPU significantly outperforms existing prompt-based methods across seven diverse tasks in the LaMP benchmark. Further studies reveal OPPU’s enhanced capabilities in handling user behavior shifts, modeling users at different activity levels, maintaining robustness across various user history formats, and displaying versatility with different PEFT methods.

Problem

• Personalization in large language models
• Addressing privacy and customization limitations
• Handling dynamic user behavior patterns

Innovation

• Personalized PEFT modules
• User-specific behavior storage
• Integrates parametric and non-parametric knowledge