Weak-to-Strong Learning in Decision Making

2026-07-20

AI Summary

This work proposes a decision-aware weak-to-strong (W2S) learning framework for settings where labeled data are scarce but contextual features are abundant. The approach first trains a weak model on limited labeled data and then leverages its predictions as soft supervision signals to train a stronger model on a large pool of unlabeled data, explicitly optimizing downstream decision performance. A non-asymptotic upper bound on the excess decision risk of W2S is established, revealing that when the relevant feature dimensions shared between the weak and strong models are sufficiently low-dimensional, unlabeled data effectively mitigate teacher error and enhance decision-making accuracy. Empirical evaluations on both a synthetic newsvendor problem and a real-world content moderation task demonstrate that the proposed method significantly outperforms strong-model-only baselines under specified conditions.

Abstract

Many operational decisions rely on predictive models that estimate uncertain outcomes conditional on observable contexts. Training such models, however, often faces a fundamental data asymmetry: labeled outcomes are scarce or costly to obtain, while contextual covariates are abundant. Motivated by this data asymmetry, we develop a decision-aware weak-to-strong (W2S) framework that leverages both labeled and unlabeled data to improve contextual stochastic optimization. Specifically, we first train a weak model using limited labeled data and then use it to generate predicted outcome distributions on unlabeled contexts. These distributions provide soft supervision for training a strong model. We establish a non-asymptotic upper bound on the excess decision risk of W2S and a complementary lower bound for a strong-only benchmark. Their comparison yields explicit sufficient conditions under which W2S improves downstream decision performance. The key quantity is the correlation dimension between the weak and strong feature representations: when it is small, abundant unlabeled data reduce the effect of teacher errors along non-overlapping directions. A synthetic newsvendor experiment and a comment moderation experiment based on real-world data provide empirical evidence consistent with the theory.

Problem

• data asymmetry
• weak-to-strong learning
• contextual stochastic optimization
• decision-aware learning
• unlabeled data

Innovation

• weak-to-strong learning
• decision-aware learning
• stochastic optimization
• semi-supervised learning
• excess decision risk