REPREC: Representation Driven Parameter-Efficient Recommendation System

2026-07-24

AI Summary

This work addresses the high training and deployment costs of existing large language model (LLM)-based sequential recommendation methods, which often rely on fine-tuning, auxiliary modules, or long item-level historical conditioning. To overcome these limitations, we propose a lightweight modular framework that maps fixed-dimensional user embeddings to a small set of learnable soft prompt tokens via a user representation alignment mechanism. Only a lightweight MLP injector is trained, while both the pretrained sequential encoder and the LLM remain entirely frozen—eliminating the need to modify any pretrained components. The approach significantly enhances recommendation performance in low-data regimes without sacrificing efficiency. Extensive experiments demonstrate consistent superiority over LoRA across multiple benchmarks, achieving an average 1.51× faster training speed and retaining 85–100% of full-performance levels under short-prompt training and long-context evaluation, with notable gains for both ordinary and core users.

Abstract

Large language models (LLMs) have been applied to sequential recommendation by formulating it as a natural language task. Previous work has improved personalization by incorporating collaborative and sequential signals through input conditioning or LLM fine-tuning. However, existing approaches often rely on one or more of the following: LLM fine-tuning, additional architectural modules, representation distillation, or item-level conditioning over long interaction histories, increasing training complexity and deployment cost. We propose REPREC, a lightweight framework that reformulates LLM-based sequential recommendation through lightweight user representation alignment. REPREC maps a fixed-size user embedding from a frozen sequential encoder into a small set of learned soft tokens through a lightweight MLP injector that conditions a frozen LLM, leaving both pretrained backbones unchanged while training only the injector. We conducted exhaustive experiments on multiple benchmark datasets and demonstrate that REPREC consistently outperforms LoRA while remaining compatible with different pretrained sequential encoders and LLM backbones, enabling a modular and production-friendly recommendation pipeline without modifying either pretrained component. The gains are particularly pronounced for casual and core users across all datasets, highlighting REPREC's effectiveness in low-data regimes. Finally, when trained on short prompt histories and evaluated with longer contexts, REPREC maintains 85-100% of LoRA's performance while reducing per-epoch training time by an average of 1.51X, demonstrating an effective balance between recommendation quality and computational efficiency for production deployment. The code is available at https://github.com/phdbotcode/REPREC

Problem

• sequential recommendation
• large language models
• parameter efficiency
• training complexity
• deployment cost

Innovation

• parameter-efficient learning
• frozen LLM
• user representation alignment
• soft prompting
• sequential recommendation