Latency-Optimal Adaptive Split Inference for Privacy-Preserving Cloud-Edge-End Collaboration

2026-08-02

AI Summary

This work addresses the challenge of performing efficient batch inference on resource-constrained IoT devices while preserving data privacy. To this end, the authors propose an adaptive cloud-edge-device collaborative split inference framework, wherein the device—acting as a trusted anchor—executes the initial plaintext portion of the model and protects intermediate activations using fully homomorphic encryption (FHE), while the edge and cloud execute their assigned model segments solely on ciphertext. The approach uniquely formulates collaborative encrypted inference as a joint selection problem of device-side split points and edge-side termination points, supporting both convolution-level and block-level fine-grained partitioning. By jointly optimizing computation, encryption, and communication overheads, the framework minimizes end-to-end latency. Experiments on CIFAR-10 and PathMNIST show that the convolution-level scheme achieves 12.9× and 3.9× speedups over pure-cloud FHE and block-level variants, respectively, with no accuracy loss, yielding average latencies of 1033.28 and 1023.43 seconds per sample.

Abstract

Internet of Things (IoT) end devices are increasingly expected to support privacy-sensitive batch inference, yet their limited computational resources often make full local execution of convolutional neural networks impractical. This paper presents a latency-optimal adaptive split inference framework for privacy-preserving cloud-edge-end collaboration. The end device acts as the trust anchor, executes the plaintext model prefix, encrypts the split activation using fully homomorphic encryption (FHE), and keeps the secret key locally, while the edge and cloud execute assigned model segments only on FHE ciphertexts. We formulate collaborative encrypted inference as a split-pair selection problem over an end-side split point and an edge-side termination point. The proposed planner jointly models plaintext prefix execution, encryption, communication, edge-side FHE execution, and cloud-side FHE completion, and supports both convolution-level and block-level split granularities. Experiments on CIFAR-10 and PathMNIST show that the proposed convolution-level collaborative scheme achieves amortized end-to-end speedups of approximately 12.9 times over full-cloud FHE and 3.9 times over the block-level alternative, while preserving the corresponding plaintext-model accuracy. Including modeled communication, the amortized latencies are 1033.279 s/sample on CIFAR-10 and 1023.429 s/sample on PathMNIST.

Problem

• privacy-preserving inference
• cloud-edge-end collaboration
• latency optimization
• split inference
• IoT devices

Innovation

• split inference
• fully homomorphic encryption
• cloud-edge-end collaboration
• latency optimization
• privacy-preserving inference