Prophet Inequalities for Bandits, Cabinets, and DAGs

2025-02-13

AI Summary

This paper studies sequential decision-making with exploration costs: a decision-maker sequentially inspects $n$ options, each modeled as a Markov search process (i.e., an undiscounted MDP over a directed acyclic graph), incurring inspection costs; ultimately, a subset satisfying combinatorial constraints (e.g., matroid) is selected to maximize total reward minus total cost. As this problem is NP-hard, the paper introduces the first universal prophet inequality framework for combinatorial Markov search. It provides a computationally efficient $(1/2 - varepsilon)$-approximation algorithm under arbitrary matroid constraints. Furthermore, it designs an incentive-compatible mechanism achieving constant-price equilibria and constant price of anarchy (PoA). These results unify and resolve classical special cases—including the non-obligatory inspection variant of the Pandora’s box problem—thereby significantly generalizing prior work on stochastic probing and optimal search.

Technology Category

• Search and Optimization: Combinatorial Optimization
• Constraint Satisfaction and Optimization: Search
• Multiagent Systems: Mechanism Design

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

A decisionmaker faces $n$ alternatives, each of which represents a potential reward. After investing costly resources into investigating the alternatives, the decisionmaker may select one, or more generally a feasible subset, and obtain the associated reward(s). The objective is to maximize the sum of rewards minus total costs invested. We consider this problem under a general model of an alternative as a"Markov Search Process,"a type of undiscounted Markov Decision Process on a finite acyclic graph. Even simple cases generalize NP-hard problems such as Pandora's Box with nonobligatory inspection. Despite the apparently adaptive and interactive nature of the problem, we prove optimal prophet inequalities for this problem under a variety of combinatorial constraints. That is, we give approximation algorithms that interact with the alternatives sequentially, where each must be fully explored and either selected or else discarded before the next arrives. In particular, we obtain a computationally efficient $frac{1}{2}-epsilon$ prophet inequality for Combinatorial Markov Search subject to any matroid constraint. This result implies incentive-compatible mechanisms with constant Price of Anarchy for serving single-parameter agents when the agents strategically conduct independent, costly search processes to discover their values.

Problem

• Maximize rewards minus costs
• Markov Search Process model
• Prophet inequalities for constraints

Innovation

• Markov Search Process
• prophet inequalities
• matroid constraint