Efficient Homomorphically Encrypted Convolutional Neural Network Without Rotation

2024-09-08

arXiv.org

AI Summary

To address the high computational latency and communication overhead in homomorphic encryption (HE)-based private inference for convolutional neural networks (CNNs) and fully connected (FC) layers—primarily caused by costly ciphertext rotations—this paper proposes the first rotation-free HE-CNN architecture. Our method eliminates rotations entirely via a channel-agnostic filter coefficient packing scheme, integrated within a server-client collaborative computation paradigm. We employ polynomial-ring-based fully homomorphic encryption (FHE), augmented with customized data/weight packing, lightweight client-side preprocessing, and efficient ciphertext multiplication on the server. Evaluated on CIFAR-10 and CIFAR-100, our approach reduces end-to-end inference time for Conv and FC layers by 15.5% and cuts client-server communication volume by over 50%, while preserving security guarantees, model accuracy, and practical efficiency.

Technology Category

• Machine Learning: Privacy
• Computer Vision: Learning & Optimization for CV
• Search and Optimization: Non-convex Optimization

Application Category

• Security and Privacy: Applications of cryptography
• Responsible Web: Data and user privacy-enhancing technologies for the Web
• User Modeling, Personalization and Recommendation: User privacy protection in personalized systems

Abstract

Privacy-preserving neural network (NN) inference can be achieved by utilizing homomorphic encryption (HE), which allows computations to be directly carried out over ciphertexts. Popular HE schemes are built over large polynomial rings. To allow simultaneous multiplications in the convolutional (Conv) and fully-connected (FC) layers, multiple input data are mapped to coefficients in the same polynomial, so are the weights of NNs. However, ciphertext rotations are necessary to compute the sums of products and/or incorporate the outputs of different channels into the same polynomials. Ciphertext rotations have much higher complexity than ciphertext multiplications and contribute to the majority of the latency of HE-evaluated Conv and FC layers. This paper proposes a novel reformulated server-client joint computation procedure and a new filter coefficient packing scheme to eliminate ciphertext rotations without affecting the security of the HE scheme. Our proposed scheme also leads to substantial reductions on the number of coefficient multiplications needed and the communication cost between the server and client. For various plain-20 classifiers over the CIFAR-10/100 datasets, our design reduces the running time of the Conv and FC layers by 15.5% and the communication cost between client and server by more than 50%, compared to the best prior design.

Problem

• Eliminate ciphertext rotations in homomorphic encryption for neural networks
• Reduce computational complexity of encrypted Conv and FC layers
• Decrease communication costs between server and client

Innovation

• Homomorphically encrypted CNN without rotations
• Server-client joint computation procedure
• New filter coefficient packing scheme