Efficient Inference for Covariate-adjusted Bradley-Terry Model with Covariate Shift

2025-03-24

AI Summary

This paper addresses pairwise comparison under covariate distribution shift, proposing a robust statistical inference framework for estimating players’ overall abilities. Methodologically, it integrates covariate shift modeling with the Bradley–Terry model for the first time; defines identifiable strength parameters via KL projection—without requiring strong parametric assumptions—and constructs a semiparametric efficient estimator that accommodates partially observed pairs. The estimator leverages double robustness and machine learning–based nuisance function estimation (e.g., neural networks, random forests). Theoretically, it is proven to be asymptotically normal; simulations confirm high-coverage, accurate confidence intervals. Empirically, the method is applied to human preference evaluation of large language models, demonstrating significantly improved generalization reliability and interpretability in cross-distribution settings.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Game Theory and Economic Paradigms: Adversarial Learning
• Reasoning under Uncertainty: Relational Probabilistic Models

Application Category

• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions

Abstract

We propose a general framework for statistical inference on the overall strengths of players in pairwise comparisons, allowing for potential shifts in the covariate distribution. These covariates capture important contextual information that may impact the winning probability of each player. We measure the overall strengths of players under a target distribution through its Kullback-Leibler projection onto a class of covariate-adjusted Bradley-Terry model. Consequently, our estimands remain well-defined without requiring stringent model assumptions. We develop semiparametric efficient estimators and corresponding inferential procedures that allow for flexible estimation of the nuisance functions. When the conditional Bradley-Terry assumption holds, we propose additional estimators that do not require observing all pairwise comparisons. We demonstrate the performance of our proposed method in simulation studies and apply it to assess the alignment of large language models with human preferences in real-world applications.

Problem

• Infer player strengths in pairwise comparisons with covariate shifts
• Estimate strengths under target distribution using KL projection
• Develop efficient estimators for covariate-adjusted Bradley-Terry models

Innovation

• Covariate-adjusted Bradley-Terry model framework
• Kullback-Leibler projection for strength estimation
• Semiparametric efficient estimators with flexible nuisance functions