From Reviews to Actionable Insights: An LLM-Based Approach for Attribute and Feature Extraction

2025-10-18

AI Summary

This study addresses the challenge of distilling interpretable managerial insights from large-scale customer reviews. Methodologically, it introduces a novel analytical framework that explicitly distinguishes *customer-perceived attributes* from *managerially actionable features*, integrating marketing theory with operational requirements; it leverages large language models enhanced by multi-strategy prompt engineering and validates model reliability through human annotation agreement and predictive validity testing. Applied to 20,000 Starbucks reviews, the approach achieves high inter-rater agreement with human coders (Cohen’s κ > 0.85) and accelerates processing by 180× compared to manual coding. Simulated intervention analysis further demonstrates that prioritizing empirically identified service improvement levers yields an estimated annual revenue uplift of 1–2% per store. The work thus advances automated, theory-grounded, and operationally relevant sentiment analytics for service management.

Technology Category

• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models
• Data Mining & Knowledge Management: Conversational Systems for Recommendation & Retrieval
• Machine Learning: Learning Preferences or Rankings

Application Category

• Web Mining and Content Analysis: Sentiment analysis and opinion mining
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

This research proposes a systematic, large language model (LLM) approach for extracting product and service attributes, features, and associated sentiments from customer reviews. Grounded in marketing theory, the framework distinguishes perceptual attributes from actionable features, producing interpretable and managerially actionable insights. We apply the methodology to 20,000 Yelp reviews of Starbucks stores and evaluate eight prompt variants on a random subset of reviews. Model performance is assessed through agreement with human annotations and predictive validity for customer ratings. Results show high consistency between LLMs and human coders and strong predictive validity, confirming the reliability of the approach. Human coders required a median of six minutes per review, whereas the LLM processed each in two seconds, delivering comparable insights at a scale unattainable through manual coding. Managerially, the analysis identifies attributes and features that most strongly influence customer satisfaction and their associated sentiments, enabling firms to pinpoint "joy points," address "pain points," and design targeted interventions. We demonstrate how structured review data can power an actionable marketing dashboard that tracks sentiment over time and across stores, benchmarks performance, and highlights high-leverage features for improvement. Simulations indicate that enhancing sentiment for key service features could yield 1-2% average revenue gains per store.

Problem

• Extracting product attributes and sentiments from reviews
• Distinguishing perceptual attributes from actionable features
• Enabling scalable customer insights for business improvements

Innovation

• LLM extracts attributes and sentiments from reviews
• Framework distinguishes perceptual attributes from actionable features
• Automated processing enables scalable marketing dashboard insights