Resilient Peer-to-peer Learning based on Adaptive Aggregation

2025-01-08

AI Summary

Addressing the robustness challenge in decentralized peer-to-peer (P2P) collaborative learning under concurrent non-i.i.d. data and malicious attacks, this paper proposes an adaptive aggregation-based resilient learning framework. Methodologically, it introduces a novel loss computation mechanism that jointly leverages neighbors’ models and local private data—enabling dynamic, privacy-preserving weight generation without raw data sharing. The framework integrates adaptive weighted aggregation, distributed optimization, and adversarially robust training. Its convergence is rigorously established under non-convex and non-i.i.d. settings via non-convex optimization theory. Extensive experiments across three representative machine learning tasks and diverse adversarial attack scenarios demonstrate that the proposed framework consistently outperforms existing P2P learning methods, achieving average accuracy improvements of 5.2%–12.7%. Key contributions include: (i) the first neighbor-model-and-local-data co-driven adaptive aggregation scheme; (ii) formal convergence guarantees for decentralized learning under non-i.i.d. and non-convex conditions; and (iii) simultaneous enhancement of privacy preservation and adversarial robustness.

Technology Category

Machine Learning: Adversarial Learning & RobustnessMultiagent Systems: Adversarial AgentsGame Theory and Economic Paradigms: Adversarial Learning

Application Category

Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingResponsible Web: Data and user privacy-enhancing technologies for the WebUser Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Collaborative learning in peer-to-peer networks offers the benefits of distributed learning while mitigating the risks associated with single points of failure inherent in centralized servers. However, adversarial workers pose potential threats by attempting to inject malicious information into the network. Thus, ensuring the resilience of peer-to-peer learning emerges as a pivotal research objective. The challenge is exacerbated in the presence of non-convex loss functions and non-iid data distributions. This paper introduces a resilient aggregation technique tailored for such scenarios, aimed at fostering similarity among peers' learning processes. The aggregation weights are determined through an optimization procedure, and use the loss function computed using the neighbor's models and individual private data, thereby addressing concerns regarding data privacy in distributed machine learning. Theoretical analysis demonstrates convergence of parameters with non-convex loss functions and non-iid data distributions. Empirical evaluations across three distinct machine learning tasks support the claims. The empirical findings, which encompass a range of diverse attack models, also demonstrate improved accuracy when compared to existing methodologies.

Problem

• Learning Network Resilience
• Data Environment
• Malicious Participants

Innovation

• Adaptive Peer Learning
• Robustness Enhancement
• Privacy Protection