Constrained Multi-Objective Reinforcement Learning with Max-Min Criterion

2026-05-29

AI Summary

This work addresses the challenge of simultaneously achieving fairness and satisfying hard constraints in multi-objective reinforcement learning (MORL). It proposes the first unified framework that integrates max-min fairness with explicit constraints, offering theoretical guarantees for policy convergence and introducing an optimization algorithm that effectively balances fairness trade-offs while ensuring constraint satisfaction. The key contribution lies in the novel incorporation of max-min fairness and hard constraints into the MORL paradigm, establishing a rigorous theoretical foundation for this integration. Empirical validation across diverse simulation environments—including building thermal control, multi-objective motion control, and low-carbon transportation—demonstrates the method’s efficacy in jointly optimizing fairness and constraint adherence.

Abstract

Multi-Objective Reinforcement Learning (MORL) extends standard RL by optimizing policies with respect to multiple, often conflicting, objectives. While max-min MORL has emerged as an effective approach for promoting fairness, its applicability remains limited, particularly when constraints must be incorporated. In this paper, we propose a MORL framework that integrates the max-min criterion with explicit constraint satisfaction. We establish a theoretical foundation for the proposed framework and validate the resulting algorithm through convergence analysis and experiments in tabular settings. We further demonstrate the practical relevance of our approach in simulated building thermal control, multi-objective locomotion control, and greenhouse-gas-emission-aware traffic management. Across these domains, our method effectively balances fairness and constraint satisfaction in multi-objective decision-making.

Problem

• Constrained Multi-Objective Reinforcement Learning
• Max-Min Criterion
• Fairness
• Constraint Satisfaction
• Multi-Objective Decision-Making

Innovation

• Constrained Multi-Objective Reinforcement Learning
• Max-Min Fairness
• Constraint Satisfaction
• Multi-Objective Decision-Making
• Reinforcement Learning