Exploiting Expertise of Non-Expert and Diverse Agents in Social Bandit Learning: A Free Energy Approach

2026-03-12

AI Summary

This work addresses the challenge in social multi-armed bandit settings where agents struggle to effectively leverage behavioral information from non-expert or heterogeneous peers without access to others’ reward signals. The authors propose a free-energy-based social learning algorithm that, for the first time, enables agents to automatically evaluate and integrate policies from relevant peers in the absence of observed rewards, without requiring prior knowledge or predefined social norms. By modeling interactions in policy space, the method selectively filters useful information and demonstrates significant performance gains over existing approaches in environments comprising experts, non-experts, and even random agents. Theoretically, the algorithm maintains a logarithmic regret bound and guarantees convergence, offering both empirical effectiveness and formal performance assurances.

Technology Category

• Machine Learning: Online Learning & Bandits
• Multiagent Systems: Multiagent Learning
• Search and Optimization: Learning to Search

Application Category

• Economics, Online Markets and Human Computation: Social networks and social learning
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

Personalized AI-based services involve a population of individual reinforcement learning agents. However, most reinforcement learning algorithms focus on harnessing individual learning and fail to leverage the social learning capabilities commonly exhibited by humans and animals. Social learning integrates individual experience with observing others' behavior, presenting opportunities for improved learning outcomes. In this study, we focus on a social bandit learning scenario where a social agent observes other agents' actions without knowledge of their rewards. The agents independently pursue their own policy without explicit motivation to teach each other. We propose a free energy-based social bandit learning algorithm over the policy space, where the social agent evaluates others' expertise levels without resorting to any oracle or social norms. Accordingly, the social agent integrates its direct experiences in the environment and others' estimated policies. The theoretical convergence of our algorithm to the optimal policy is proven. Empirical evaluations validate the superiority of our social learning method over alternative approaches in various scenarios. Our algorithm strategically identifies the relevant agents, even in the presence of random or suboptimal agents, and skillfully exploits their behavioral information. In addition to societies including expert agents, in the presence of relevant but non-expert agents, our algorithm significantly enhances individual learning performance, where most related methods fail. Importantly, it also maintains logarithmic regret.

Problem

• social bandit learning
• non-expert agents
• social learning
• reinforcement learning
• policy evaluation

Innovation

• social bandit learning
• free energy principle
• non-expert agents
• policy integration
• logarithmic regret