History Matters: Meta-policy Delegation with Heterogeneous Multi-agent Reinforcement Learning

2026-08-04

AI Summary

This work addresses the challenge of enabling efficient collaboration among heterogeneous multi-agent systems under resource constraints while minimizing total execution cost. To this end, the authors propose a dynamic task delegation mechanism grounded in multi-agent reinforcement learning (MARL). The approach introduces a history-dependent policy framework that overcomes the limitation of conventional MARL methods, which rely solely on current system states, and incorporates a multidimensional monetary incentive scheme to facilitate flexible task delegation among agents with diverse capabilities and cost structures. Experimental results demonstrate that the proposed method significantly enhances collaborative efficiency and effectively reduces overall system cost in fully observable environments.

Abstract

AI agents are expected to play an increasingly important role in future decision-making systems. In this paper, we consider collaborative systems composed of heterogeneous multi-agent systems (MAS), where their members have different capabilities and operating costs. We study how agents can delegate tasks to one another so that certain research tasks can be completed effectively under resource-constrained scenarios. We first develop a multi-agent reinforcement learning-based (MARL) delegation training that enables agents to make sequential delegation decisions while minimizing the total execution cost. We then extend this approach to MARL with prescribed delegation topologies. Furthermore, we introduce two new frameworks for collaboration and delegation in multi-agent systems. The first framework proposes that an agent's policy depends not only on the current state of the underlying Markov decision process but also on the interaction history, including previous joint actions. This history-dependent formulation can improve coordination even in fully observable environments, where conventional MARL methods typically restrict policies to depend only on the current state. The second framework proposes a novel, potentially multi-dimensional monetary mechanism to facilitate the collaboration and delegation for MAS.

Problem

• heterogeneous multi-agent systems
• task delegation
• resource-constrained scenarios
• multi-agent reinforcement learning
• collaborative decision-making

Innovation

• history-dependent policy
• heterogeneous multi-agent reinforcement learning
• task delegation
• monetary mechanism
• collaborative MARL