From What to Why: Thought-Space Recommendation with Small Language Models

2025-10-08

AI Summary

Large language models (LLMs) incur prohibitive inference costs and deployment challenges, while small language models (SLMs) remain underutilized for semantic reasoning in recommendation tasks. Method: We propose a novel SLM-based “reasoning space” recommendation framework that treats natural-language rationales—supervised from user interaction histories—as direct learning signals. It jointly models behavioral patterns and semantic motivations to construct a unified, generalizable representation space, moving beyond conventional ID- or sequence-based modeling. The approach integrates self-supervised learning with semantic embedding to enable efficient cross-domain recommendation. Contribution/Results: Extensive experiments demonstrate significant improvements over ID-based baselines, collaborative filtering, and LLM-based sequential recommenders across multiple benchmarks. Moreover, the learned representations exhibit strong transferability to downstream reasoning-intensive tasks, such as inference-oriented question answering.

Technology Category

• Knowledge Representation and Reasoning: Reasoning with Beliefs
• Cognitive Modeling & Cognitive Systems: Conceptual Inference and Reasoning
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Large Language Models (LLMs) have advanced recommendation capabilities through enhanced reasoning, but pose significant challenges for real-world deployment due to high inference costs. Conversely, while Small Language Models (SLMs) offer an efficient alternative, their reasoning capabilities for recommendation remain underexplored. Existing systems often use natural language rationales merely as unsupervised descriptive text, failing to harness their full potential as learning signals. In this work our main idea is to create a common understanding of user and items across multiple domains called Thought Space with SLMs instead of using LLMs' distilled knowledge. To that end we propose PULSE (Preference Understanding by Latent Semantic Embeddings), a framework that treats SLM-generated rationales as director learning signals, supervising them with interaction histories to jointly model user actions (what) and their semantic drivers (why). Existing methods consider only interactions such as sequences and embeddings, whereas PULSE treats rationales as first-class signals, this novel design yields embeddings that are more robust and generalizable. Extensive experiments demonstrate that PULSE outperforms leading ID, Collaborative Filtering (CF), and LLM-based sequential recommendation models across multiple benchmark datasets. Furthermore, PULSE exhibits superior transferability in cross-domain recommendation and demonstrates strong performance on downstream tasks such as reasoning-oriented question answering. Our code is available href{https://anonymous.4open.science/r/Thinking_PULSE-0FC5/README.md}{here}.

Problem

• Addressing high inference costs of LLMs in recommendation systems
• Enhancing reasoning capabilities of small language models for recommendations
• Leveraging rationales as learning signals rather than descriptive text

Innovation

• Uses small language models for efficient recommendation systems
• Creates Thought Space to understand user actions and motivations
• Treats rationales as primary learning signals for embeddings