Scale-free adaptive planning for deterministic dynamics & discounted rewards

2026-04-20

AI Summary

This work proposes Platypoos, a tuning-free planning algorithm for stochastic discounted-reward environments where rewards are unbounded, dynamics are deterministic, and the optimal value function is unknown. Platypoos requires no prior knowledge of the reward scale or smoothness and adapts automatically to varying discount factors and reward magnitudes. It is the first algorithm to achieve adaptivity to both unknown reward scale and smoothness while attaining a tight sample complexity upper bound—matching the theoretical lower bound up to constant factors—across a broad range of discount factors. Empirical evaluations demonstrate that Platypoos significantly outperforms existing methods in settings with unknown reward scales.

Technology Category

Reasoning under Uncertainty: Stochastic OptimizationSearch and Optimization: Sampling/Simulation-based SearchPlanning, Routing, and Scheduling: Learning for Planning and Scheduling

Application Category

Economics, Online Markets and Human Computation: Economics and fairness of platforms and recommendation systemsGraph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphsResponsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

We address the problem of planning in an environment with deterministic dynamics and stochastic rewards with discounted returns. The optimal value function is not known, nor are the rewards bounded. We propose Platypoos, a simple scale-free planning algorithm that adapts to the unknown scale and smoothness of the reward function. We provide a sample complexity analysis for Platypoos that improves upon prior work and holds simultaneously over a broad range of discount factors and reward scales, without the algorithm knowing them. We also establish a matching lower bound showing our analysis is optimal up to constants.

Problem

• deterministic dynamics
• stochastic rewards
• discounted rewards
• scale-free planning
• unknown reward scale

Innovation

• scale-free planning
• adaptive algorithm
• deterministic dynamics
• discounted rewards
• sample complexity