Multi-Hypothesis Prediction for Portfolio Optimization: A Structured Ensemble Learning Approach to Risk Diversification

2025-01-07

AI Summary

This paper addresses insufficient risk diversification in portfolio optimization by proposing a structured ensemble learning framework based on multi-hypothesis prediction, unifying asset selection and weight optimization within a prediction-to-optimization pipeline subject to diversity constraints. Its key contributions include: (i) explicitly linking ensemble loss decomposition theory to portfolio diversification; (ii) introducing a pre-screening mechanism that dynamically balances predictive accuracy against structural diversity; and (iii) constructing a parameterized prediction set with controllable diversity and a supervised ensemble combiner (e.g., equal-weighted aggregation under squared loss). Empirical evaluation across over two decades of S&P 500 constituents and a global bond dataset comprising 1,300 instruments demonstrates significantly expanded achievable diversification bounds. The framework delivers robust, state-of-the-art performance in both single-period and multi-period portfolio allocation tasks.

Technology Category

• Machine Learning: Ensemble Methods
• Reasoning under Uncertainty: Stochastic Optimization
• Search and Optimization: Learning to Search

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Fairness, privacy, and diversity in economic environments
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

This work proposes a unified framework for portfolio allocation, covering both asset selection and optimization, based on a multiple-hypothesis predict-then-optimize approach. The portfolio is modeled as a structured ensemble, where each predictor corresponds to a specific asset or hypothesis. Structured ensembles formally link predictors' diversity, captured via ensemble loss decomposition, to out-of-sample risk diversification. A structured data set of predictor output is constructed with a parametric diversity control, which influences both the training process and the diversification outcomes. This data set is used as input for a supervised ensemble model, the target portfolio of which must align with the ensemble combiner rule implied by the loss. For squared loss, the arithmetic mean applies, yielding the equal-weighted portfolio as the optimal target. For asset selection, a novel method is introduced which prioritizes assets from more diverse predictor sets, even at the expense of lower average predicted returns, through a diversity-quality trade-off. This form of diversity is applied before the portfolio optimization stage and is compatible with a wide range of allocation techniques. Experiments conducted on the full S&P 500 universe and a data set of 1.300 global bonds of various types over more than two decades validate the theoretical framework. Results show that both sources of diversity effectively extend the boundaries of achievable portfolio diversification, delivering strong performance across both one-step and multi-step allocation tasks.

Problem

• Proposes a unified framework for portfolio allocation and optimization
• Links predictors' diversity to risk diversification via ensemble learning
• Introduces asset selection prioritizing diversity over predicted returns

Innovation

• Structured ensemble learning for risk diversification
• Parametric diversity control in predictor output
• Diversity-quality trade-off in asset selection