Algebraic Cryptanalytic Extraction on Hard-Label Neural Networks

2026-08-06

AI Summary

This work addresses the high computational cost and inefficiency of existing hard-label model extraction attacks that rely on singular value decomposition (SVD). By reframing the problem from an algebraic perspective, the authors propose the Approximate Signature Vector (ASV) method, which replaces SVD with efficient inner product operations for clustering, substantially reducing computational complexity. The approach achieves the first effective parameter extraction against convolutional neural networks incorporating max-pooling layers—such as LeNet-5—and demonstrates superior performance on a 64-64×4-10 fully connected network, significantly accelerating clustering and overall extraction efficiency compared to current methods. This study fills a critical gap in hard-label extraction for CNNs with pooling operations and establishes a novel paradigm grounded in algebraic formulations.

Abstract

Although the state-of-the-art neural network model extraction attack in the hard-label setting by Carlini et al. at EUROCRYPT 2025 has polynomial-time complexity in theory, its dual-point clustering relies on singular value decomposition (SVD) with a time complexity of $\mathcal{O}(n^2 \cdot (d^{(k)})^3)$, resulting in huge runtime in practice. To address this computational bottleneck, this work transforms Carlini et al.'s geometric-view hard-label attack into an algebraic framework, and proposes a novel Approximate Signature Vector (ASV) method to achieve efficient parameter extraction on Fully Connected Neural Networks (FCNNs) by leveraging two key observations: high-dimensional random vectors are nearly orthogonal, and neurons in practical DNNs tend to learn disentangled features. The proposed ASV method replaces SVD-based rank checking with simple inner-product operations, reducing the clustering complexity to $\mathcal{O}(n \cdot (d^{(k)})^3)$ on average. Furthermore, this paper presents the first model extraction attack against hard-label max-pooling Convolutional Neural Networks (CNNs) by proposing an advanced ASV method with a kernel-centric clustering scheme instead of the neuron-centric clustering, which fully exploits the property of weight sharing in convolutions and fills the cryptanalysis gap. Experiments on a 64-64$\times$4-10 FCNN and LeNet-5 (CNN) with max pooling demonstrate that our ASV method drastically cuts clustering time, and improves the overall efficiency in the model extraction.

Problem

• model extraction
• hard-label attacks
• computational bottleneck
• convolutional neural networks
• fully connected neural networks

Innovation

• Approximate Signature Vector
• Hard-Label Model Extraction
• Algebraic Cryptanalysis
• Kernel-Centric Clustering
• Fully Connected Neural Networks