Same Performance, Hidden Bias: Evaluating Hypothesis- and Recommendation-Driven AI

2026-03-16

Technology Category

Philosophy and Ethics of AI: Bias, Fairness & EquityHumans and AI: User Experience and UsabilityKnowledge Representation and Reasoning: Preferences

Application Category

User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating successSearch and Retrieval-Augmented AI: Web evaluation methodologies and metricsEconomics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systems

Abstract

The HCI community commonly evaluates decision support systems based on whether they improve task performance or promote appropriate user reliance. In this work, we look beyond decision outcomes to examine the process through which users develop decision-making strategies. Through a web-based experiment (N = 290) comparing recommendation-driven and hypothesis-driven interaction designs, and using Signal Detection Theory as a theoretical framework, we show that even when performance remains identical, recommendation-driven designs lower participants' thresholds for sufficient evidence and introduce a "hidden bias" in their judgments, resulting in a shifted distribution of errors. Furthermore, we find that experts are just as susceptible to these systemic shifts as novices. We conclude by advocating for a shift in focus: prioritizing decision processes and the preservation of stable evidence standards over performance and reliance alone.

Similar Papers

AI AI Bias: Large Language Models Favor Their Own Generated Content

2024-07-09arXiv.orgCitations: 5