🤖 AI Summary
This work studies a class of combinatorial online learning problems—such as optimal binary search trees (BSTs), matrix-chain multiplication, and knapsack—that admit efficient dynamic programming (DP) solutions. It focuses specifically on online BST optimization under dynamically varying access frequencies: in each round, a low-average-search-cost tree must be selected rapidly to minimize cumulative regret against the best fixed offline BST. The paper innovatively extends the Hedge and Component Hedge algorithms to DP-induced exponentially large combinatorial structures, introducing a novel weight-propagation mechanism based on subproblem decomposition, probabilistic tree construction, and a corresponding regret analysis framework. Theoretically, it achieves a sublinear regret bound—strictly superior to brute-force enumeration or greedy baselines. Moreover, the approach is generalizable, enabling natural online adaptation of diverse DP-based problems.
📝 Abstract
We consider the problem of repeatedly solving a variant of the same dynamic programming problem in successive trials. An instance of the type of problems we consider is to find a good binary search tree in a changing this http URL the beginning of each trial, the learner probabilistically chooses a tree with the $n$ keys at the internal nodes and the $n+1$ gaps between keys at the leaves. The learner is then told the frequencies of the keys and gaps and is charged by the average search cost for the chosen tree. The problem is online because the frequencies can change between trials. The goal is to develop algorithms with the property that their total average search cost (loss) in all trials is close to the total loss of the best tree chosen in hindsight for all trials. The challenge, of course, is that the algorithm has to deal with exponential number of trees. We develop a general methodology for tackling such problems for a wide class of dynamic programming algorithms. Our framework allows us to extend online learning algorithms like Hedge and Component Hedge to a significantly wider class of combinatorial objects than was possible before.