Integrity of peer-to-peer distributed LLM inference under malicious nodes

2026-07-21

AI Summary

This work addresses the threat of malicious nodes compromising inference integrity in peer-to-peer distributed execution of large language models by tampering with intermediate layer outputs. To counter this, the authors propose a threshold-free detection mechanism based on “canary” inputs—secret samples with known responses injected during inference. By monitoring distributional shifts in activation values and applying a probabilistic test to distinguish hardware-induced noise from adversarial manipulation, the method reliably identifies compromised nodes without requiring predefined thresholds. Evaluated across 408 diverse configurations, the approach achieves a perfect AUROC of 1.0, consistently ranking all malicious shards ahead of benign ones, thereby significantly enhancing the robustness and security of distributed inference.

Abstract

Peer-to-peer distributed inference executes a Large Language Model (LLM) on pooled consumer hardware by spreading its layers across many nodes. Every request passes through nodes that are owned and controlled by multiple independent parties. However, in this setting, any party can tamper with the output of its layers to corrupt the end result. Recomputing the forward pass on trusted hardware can catch this, but it introduces additional computational cost. The scientific literature includes several prior integrity-checking approaches, such as known-answer traps for image classifiers and cryptographic commitments. However, these solutions test only the exact correctness and do not account for the ordinary variation that may arise between benign nodes. In this paper, we propose a method that checks the output integrity by measuring the variation in the activations that each node passes to the next. A peer who wants to use the network selects a small set of secret canary inputs whose correct activations are known in advance and mixes them into regular traffic. Because the peers cannot tell a canary from a real query, any tampering node corrupts them as well. The deviation from the known reference then reveals malicious activity: benign nodes exhibit only minor variation from hardware-induced noise, whereas tampered nodes deviate far more. We treat the identification of malicious nodes as a probabilistic test that separates two drift distributions, without relying on a fixed threshold. We study 408 configurations with metrics and success criteria fixed before any experiment ran; the detector reaches AUROC 1.0, correctly ranking the malicious shard above every benign shard on every canary in every configuration.

Problem

• peer-to-peer
• distributed LLM inference
• integrity
• malicious nodes
• output tampering

Innovation

• canary inputs
• activation integrity
• distributed LLM inference
• malicious node detection
• probabilistic drift detection