Certified in Theory, Broken in Practice: Assumption Gaps in Cryptographic Model Certification

2026-07-23

AI Summary

This work addresses a critical limitation in existing certification schemes for encrypted machine learning models, which only verify model behavior on a fixed audit dataset and thus fail to guarantee generalization to new, identically distributed data—rendering them vulnerable to adversarial manipulation. We formally introduce, for the first time, a generalizable security definition tailored to encrypted model certification and expose fundamental assumptions underlying current zero-knowledge proof–based privacy-preserving auditing protocols that do not hold in practical deployments. To bridge this gap, we propose a unified certification framework integrating secure multi-party computation, zero-knowledge proofs, and statistical generalization theory, providing formal guarantees that audit outcomes generalize to real-world data. Empirical evaluation demonstrates that adversaries can achieve over 99% accuracy during audits while degrading true model performance to below 30%; our protocol effectively mitigates such attacks, aligning theoretical assurances with real-world robustness.

Abstract

Privacy-preserving machine learning auditing protocols allow auditors to assess models for properties such as accuracy or fairness, without revealing their internals or training data. This makes them especially attractive for auditing models deployed in sensitive domains such as healthcare or finance. For these protocols to be meaningful in real-world audit settings, though, their guarantees must reflect how the model will behave once deployed, rather than merely certifying its behavior during an audit. Existing security definitions often miss this mark: most certify model behavior only on a fixed audit dataset, without ensuring that the same guarantees generalize to other datasets drawn from the same distribution. As we show, this gap allows a model provider to attack many cryptographic model certification (CMC) schemes built on secure zero knowledge proofs (ZKP) by carefully engineering training data, resulting in models that exhibit benign behavior during an audit, but pathological behavior in practice. For example, we empirically demonstrate that an attacker can certify that a model achieves over 99% accuracy on an audit dataset, but less than 30% accuracy on fresh samples from the same distribution. To address this gap, we formalize rigorous cryptographic security notions tailored to CMC frameworks, introduce a generic protocol template, and prove that it satisfies these requirements. Our results thus offer both cautionary evidence about existing approaches and constructive guidance for designing secure, privacy-preserving ML auditing protocols.

Problem

• cryptographic model certification
• privacy-preserving machine learning
• zero-knowledge proofs
• generalization gap
• model auditing

Innovation

• cryptographic model certification
• zero-knowledge proofs
• privacy-preserving auditing
• generalization gap
• secure machine learning