Gecko: Fast Private Inference via Secure Public Encoder Offloading

2026-08-03

AI Summary

This work addresses the inefficiency of existing privacy-preserving inference schemes and the vulnerability introduced by placing public encoders outside the protection boundary, which can create feature-space shortcuts and heighten model extraction risks. The authors propose an efficient and secure privacy inference framework that freezes a pretrained backbone to extract hierarchical features, compresses feature dimensions via Fastfood random projections, and incorporates a private feature gating mechanism to block feature reuse attack paths. Only a lightweight predictor is encrypted and executed under secure multi-party computation for efficient inference. Adhering to design principles of ideal independence and information retention, the method achieves 0.4–2.2 seconds latency and ≤10.8 MB communication overhead on image and audio tasks, matching the accuracy of transfer learning baselines while empirically demonstrating that adversaries gain no significant advantage from reusing the public encoder.

Abstract

Private inference protects both user inputs and server models during neural network inference, but existing solutions remain too slow for practical deployment. This motivates recent efforts to run a public encoder, such as a pretrained backbone, outside the protection boundary and evaluate only a small private predictor cryptographically. While appealing for efficiency, this design is not inherently secure: naively offloading a public encoder may create a feature-space shortcut: an extraction adversary may learn the remaining private predictor's feature-to-output mapping more easily than the original model's input-to-output behavior. We present Gecko, designed to limit this additional risk while retaining a compact encrypted predictor. We leverage a frozen backbone that contributes hierarchical features, fixed Fastfood projections that compress them, and private feature gating that prepares them for prediction. We formalize ideal independence and information-preservation conditions as design guidance, then separately evaluate component-reuse extraction attacks. Across image and audio tasks, Gecko achieves 0.4-2.2 second inference with at most 10.8 MB communication and accuracy comparable to transfer-learning baselines. Under the evaluated attacks, reusing the offloaded public encoder provides no significant advantage to model-extraction adversaries. Source code and a demo are available at https://github.com/CassiniHuy/gecko-infer.

Problem

• private inference
• model extraction
• feature-space shortcut
• secure offloading
• neural network privacy

Innovation

• private inference
• secure public encoder offloading
• feature-space shortcut
• Fastfood projection
• model extraction resistance