Understanding Visual Saliency of Outlier Items in Product Search

2025-03-30

AI Summary

This study investigates how visually anomalous products in e-commerce search listings influence user attention allocation and item exposure distribution through visual salience and cognitive mechanisms. It addresses two core questions: (i) the independent contribution of low-level visual features to product anomaly, and (ii) the modulatory role of high-level semantic guidance on anomaly perception in realistic shopping contexts. For the first time, the work systematically disentangles bottom-up visual contrast from top-down semantic guidance, proposing a dual-path evaluation framework integrating computational saliency models (DeepGaze, SAM), online shopping tasks, and joint analysis of reaction times and fixation durations. Results show that semantically congruent product descriptions attract attention fastest (top-down dominance), whereas anomalous items elicit significantly longer average fixation durations—confirming their robust attentional capture. This research establishes a novel paradigm and empirical foundation for understanding attention competition dynamics in two-sided e-commerce markets.

Technology Category

• Search and Optimization: Evaluation and Analysis
• Computer Vision: Visual Reasoning & Symbolic Representations
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

In two-sided marketplaces, items compete for user attention, which translates to revenue for suppliers. Item exposure, indicated by the amount of attention items receive in a ranking, can be influenced by factors like position bias. Recent work suggests that inter-item dependencies, such as outlier items in a ranking, also affect item exposure. Outlier items are items that observably deviate from the other items in a ranked list. Understanding outlier items is crucial for determining an item's exposure distribution. In our previous work, we investigated the impact of different presentational features on users' perception of outlier in search results. In this work, we focus on two key questions left unanswered by our previous work: (i) What is the effect of isolated bottom-up visual factors on item outlierness in product lists? (ii) How do top-down factors influence users' perception of item outlierness in a realistic online shopping scenario? We start with bottom-up factors and employ visual saliency models to evaluate their ability to detect outlier items in product lists purely based on visual attributes. Then, to examine top-down factors, we conduct eye-tracking experiments on an online shopping task. Moreover, we employ eye-tracking to not only be closer to the real-world case but also to address the accuracy problem of reaction time in the visual search task. Our experiments show the ability of visual saliency models to detect bottom-up factors, consistently highlighting areas with strong visual contrasts. The results of our eye-tracking experiment for lists without outliers show that despite being less visually attractive, product descriptions captured attention the fastest, indicating the importance of top-down factors. In our eye-tracking experiments, we observed that outlier items engaged users for longer durations compared to non-outlier items.

Problem

• Investigates visual saliency's role in outlier item detection
• Examines bottom-up and top-down factors affecting item outlierness
• Uses eye-tracking to study user attention in online shopping

Innovation

• Using visual saliency models for outlier detection
• Conducting eye-tracking experiments for top-down factors
• Analyzing visual contrasts and attention durations