Problem
Research questions and friction points this paper is trying to address.
E-commerce search
page-level layout
offline evaluation
secondary stack
user engagement
Language Models for Page-Level Layout Decisions in E-commerce Search
🤖 AI Summary
This study addresses the challenges of evaluating e-commerce search page layouts and the prohibitive costs associated with online A/B testing. To this end, it proposes a scalable offline evaluation method leveraging the representation layers of large language models (LLMs). Departing from conventional prompt engineering, the approach extracts deep semantic representations from LLMs to predict how secondary stack insertion positions affect user engagement. Experimental results demonstrate that this representation-based method significantly outperforms both direct prompting and feature-engineering baselines in engagement prediction. Ultimately, this work establishes an efficient and reliable offline evaluation paradigm for page-level layout decision-making.
📝 Abstract
E-commerce search pages are critical touchpoints for millions of online shoppers. While traditional search engines return a ranked list of results, modern E-commerce search pages increasingly incorporate recommender system modules -- for example, secondary stacks that surface alternative product groupings at specific positions. When introduced appropriately, secondary stacks can improve user engagement; however, suboptimal placement may disrupt browsing flow and degrade the primary results. Unlike traditional search ranking, where evaluation techniques such as interleaving are well established, evaluating page-level layout changes e.g., when and where to insert a secondary stack remains challenging without costly online A/B testing. To address this, we study offline methods for evaluating whether a given layout decision -- specifically, the inclusion of a secondary stack at a particular position -- is beneficial to users. We investigate language models as scalable evaluators by comparing direct prompt-based, prompt-derived feature, and representation-based methods. Our results show that representation-based approaches consistently outperform prompt-based judging in predicting user engagement, suggesting they provide a reliable foundation for offline layout evaluation in E-commerce search.
Innovation
Methods, ideas, or system contributions that make the work stand out.
Language Models
Page-Level Layout
E-commerce Search
Offline Evaluation
Representation-based Methods
🔎 Similar Papers
No similar papers found.
