PeReGrINE: Evaluating Personalized Review Fidelity with User Item Graph Context

2026-04-09

AI Summary

This work addresses the lack of systematic evaluation of comment fidelity within user-item graph contexts in existing personalized review generation methods. It introduces a temporal-consistency-aware bipartite graph benchmark that conditions review generation on constrained evidence—including user history, item context, and neighbor interactions—and incorporates user style parameters to model long-term preferences, enabling controllable comparisons across diverse graph retrieval settings. The study further proposes Dissonance Analysis, a novel macro-level evaluation framework that uniquely integrates user linguistic style with product consensus deviation, while jointly leveraging visual evidence and graph-structured context to systematically assess the impact of multi-source evidence on review fidelity. Experiments demonstrate that graph-derived evidence is crucial for achieving both personalization and consistency, visual cues can enhance text quality in specific scenarios, and the approach yields reproducible, high-quality reviews across multiple product categories.

Abstract

We introduce PeReGrINE, a benchmark and evaluation framework for personalized review generation grounded in graph-structured user--item evidence. PeReGrINE restructures Amazon Reviews 2023 into a temporally consistent bipartite graph, where each target review is conditioned on bounded evidence from user history, item context, and neighborhood interactions under explicit temporal cutoffs. To represent persistent user preferences without conditioning directly on sparse raw histories, we compute a User Style Parameter that summarizes each user's linguistic and affective tendencies over prior reviews. This setup supports controlled comparison of four graph-derived retrieval settings: product-only, user-only, neighbor-only, and combined evidence. Beyond standard generation metrics, we introduce Dissonance Analysis, a macro-level evaluation framework that measures deviation from expected user style and product-level consensus. We also study visual evidence as an auxiliary context source and find that it can improve textual quality in some settings, while graph-derived evidence remains the main driver of personalization and consistency. Across product categories, PeReGrINE offers a reproducible way to study how evidence composition affects review fidelity, personalization, and grounding in retrieval-conditioned language models.

Problem

• personalized review generation
• review fidelity
• user-item graph
• evidence grounding
• style consistency

Innovation

• personalized review generation
• user-item graph
• User Style Parameter
• Dissonance Analysis
• retrieval-conditioned language models