Enhance Large Language Models as Recommendation Systems with Collaborative Filtering

2025-10-17

AI Summary

Existing non-fine-tuned large language model–based recommender systems (LLM-RS) lack task-specific knowledge—particularly user-item interaction patterns—leading to suboptimal recommendation accuracy and personalization. To address this, we propose Critique-based LLM-RS: the first framework that, *without fine-tuning the LLM*, introduces an independent, collaborative-filtering–driven critic model to explicitly capture user-item interactions and generate structured critique signals that dynamically refine the LLM’s raw recommendations. This approach enables plug-and-play injection of domain-specific business knowledge without modifying LLM parameters. Extensive experiments on multiple real-world datasets demonstrate significant improvements in both accuracy (e.g., +12.3% Recall@10) and personalization. Our method establishes a new paradigm for lightweight, interpretable, and production-ready LLM-RS.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Data Mining & Knowledge Management: Recommender Systems
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

As powerful tools in Natural Language Processing (NLP), Large Language Models (LLMs) have been leveraged for crafting recommendations to achieve precise alignment with user preferences and elevate the quality of the recommendations. The existing approaches implement both non-tuning and tuning strategies. Compared to following the tuning strategy, the approaches following the non-tuning strategy avoid the relatively costly, time-consuming, and expertise-requiring process of further training pre-trained LLMs on task-specific datasets, but they suffer the issue of not having the task-specific business or local enterprise knowledge. To the best of our knowledge, none of the existing approaches following the non-tuning strategy explicitly integrates collaborative filtering, one of the most successful recommendation techniques. This study aims to fill the gap by proposing critique-based LLMs as recommendation systems (Critic-LLM-RS). For our purpose, we train a separate machine-learning model called Critic that implements collaborative filtering for recommendations by learning from the interactions between many users and items. The Critic provides critiques to LLMs to significantly refine the recommendations. Extensive experiments have verified the effectiveness of Critic-LLM-RS on real datasets.

Problem

• Enhancing LLMs for recommendations with collaborative filtering
• Addressing lack of domain knowledge in non-tuning approaches
• Integrating CF critiques to refine LLM-generated recommendations

Innovation

• Integrates collaborative filtering with LLMs
• Trains separate Critic model for recommendations
• Uses critiques to refine LLM recommendations