Backward Bayesian Outcome Weighted Learning

2026-07-31

AI Summary

This work addresses the lack of effective quantification of individualized decision uncertainty in existing methods for learning multi-stage dynamic treatment strategies. It introduces Bayesian inference into the outcome-weighted learning framework for the first time, integrating backward induction to directly learn optimal policies while propagating and quantifying uncertainty in treatment recommendations throughout the backward recursion. The proposed approach enjoys theoretical guarantees of estimation consistency and propriety, and simulation studies demonstrate its ability to accurately identify optimal dynamic treatment regimes in multi-stage settings while reliably characterizing uncertainty at the individual decision level.

Abstract

A central objective of precision medicine is learning optimal dynamic treatment regimes (DTRs) from data. Classification-based methods, like outcome weighted learning (OWL) for single-stage and backward OWL (BOWL) for multi-stage problems, leverage machine learning to directly learn optimal DTRs. However, these methods lack a natural way to quantify uncertainty in treatment decisions at the individual level. In this paper, we extend Bayesian OWL, a Bayesian reformulation of OWL, to the multi-stage setting. We call this method backward Bayesian outcome weighted learning (BBOWL). Like BOWL, our method directly learns an optimal DTR via backward induction, and unlike existing methods, our approach propagates uncertainty backward through the DTR learning process and provides uncertainty quantification of individualized treatment recommendations. We present a theoretical justification of BBOWL and verify its performance via a simulation study.

Problem

• dynamic treatment regimes
• uncertainty quantification
• precision medicine
• Bayesian learning
• individualized treatment recommendations

Innovation

• Bayesian outcome weighted learning
• dynamic treatment regimes
• uncertainty quantification
• backward induction
• precision medicine