Off-Policy Evaluation for Ranking Policies under Deterministic Logging Policies

2026-03-22

AI Summary

This work addresses the severe bias in existing off-policy evaluation methods under fully deterministic logging policies, which stems from insufficient exploration. To overcome this limitation, the paper proposes the Click-based Inverse Propensity Score (CIPS) estimator, which leverages the inherent randomness in user click behavior to construct importance weights, thereby eliminating the conventional reliance on stochasticity in the logging policy. Theoretical analysis demonstrates that CIPS achieves lower bias and controllable variance. Extensive experiments on both synthetic and real-world datasets confirm that CIPS significantly outperforms strong baseline methods, yielding substantially improved evaluation accuracy while maintaining stability.

Technology Category

• Machine Learning: Online Learning & Bandits
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Sampling/Simulation-based Search

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Security and Privacy: Large-scale security measurements

Abstract

Off-Policy Evaluation (OPE) is an important practical problem in algorithmic ranking systems, where the goal is to estimate the expected performance of a new ranking policy using only offline logged data collected under a different, logging policy. Existing estimators, such as the ranking-wise and position-wise inverse propensity score (IPS) estimators, require the data collection policy to be sufficiently stochastic and suffer from severe bias when the logging policy is fully deterministic. In this paper, we propose novel estimators, Click-based Inverse Propensity Score (CIPS), exploiting the intrinsic stochasticity of user click behavior to address this challenge. Unlike existing methods that rely on the stochasticity of the logging policy, our approach uses click probability as a new form of importance weighting, enabling low-bias OPE even under deterministic logging policies where existing methods incur substantial bias. We provide theoretical analyses of the bias and variance properties of the proposed estimators and show, through synthetic and real-world experiments, that our estimators achieve significantly lower bias compared to strong baselines, for a range of experimental settings with completely deterministic logging policies.

Problem

• Off-Policy Evaluation
• Ranking Policies
• Deterministic Logging Policies
• Inverse Propensity Score
• Click Behavior

Innovation

• Off-Policy Evaluation
• Deterministic Logging Policy
• Click-based Inverse Propensity Score
• Ranking Policy
• Importance Weighting