ProToM: Promoting Prosocial Behaviour via Theory of Mind-Informed Feedback

2025-09-05

AI Summary

In multi-agent systems, agents pursuing individual objectives often exhibit diminished collaborative intent, necessitating mechanisms that incentivize prosocial behavior—i.e., altruistic actions that do not compromise personal goals. To address this, we propose ProToM, the first framework integrating Theory of Mind (ToM) with Bayesian Inverse Planning (BIP). ProToM enables real-time inference of other agents’ goals and generates context-aware, low-overhead feedback grounded in expected utility maximization—thereby facilitating timely mutual assistance without explicit coordination. Integrated into large language and reasoning models, ProToM supports real-time interactive decision-making. Experiments in the Doors, Keys and Gems and Overcooked environments demonstrate that ProToM significantly improves task success rates and completion efficiency. Furthermore, human preference evaluations show ProToM outperforms baseline methods, validating its effectiveness and practicality in enhancing implicit collaboration.

Technology Category

• Multiagent Systems: Multiagent Planning
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Multi-Robot Systems

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Agentic search
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

While humans are inherently social creatures, the challenge of identifying when and how to assist and collaborate with others - particularly when pursuing independent goals - can hinder cooperation. To address this challenge, we aim to develop an AI system that provides useful feedback to promote prosocial behaviour - actions that benefit others, even when not directly aligned with one's own goals. We introduce ProToM, a Theory of Mind-informed facilitator that promotes prosocial actions in multi-agent systems by providing targeted, context-sensitive feedback to individual agents. ProToM first infers agents' goals using Bayesian inverse planning, then selects feedback to communicate by maximising expected utility, conditioned on the inferred goal distribution. We evaluate our approach against baselines in two multi-agent environments: Doors, Keys, and Gems, as well as Overcooked. Our results suggest that state-of-the-art large language and reasoning models fall short of communicating feedback that is both contextually grounded and well-timed - leading to higher communication overhead and task speedup. In contrast, ProToM provides targeted and helpful feedback, achieving a higher success rate, shorter task completion times, and is consistently preferred by human users.

Problem

• Promoting prosocial behavior in multi-agent systems
• Developing AI feedback for better human cooperation
• Addressing contextual and timely communication challenges

Innovation

• Uses Bayesian inverse planning to infer agent goals
• Maximizes expected utility for feedback selection
• Provides context-sensitive feedback to promote prosocial behavior