zkLLMPoT: Efficient Zero Knowledge Proof of Training for Large Language Models

2026-10-06

AI Summary

This study addresses the inherent tension between privacy preservation and the prohibitive computational overhead of full-trajectory verification in auditing large model training. To resolve this, it proposes a two-stage zero-knowledge proof protocol based on forward evaluation, leveraging Sumcheck and Lookup arguments to construct a framework that decouples certification costs from the number of training iterations. This approach supports the verification of Transformer computations and task-specific objectives without exposing model weights or accessing private data. The primary contribution is the realization of efficient, privacy-preserving training proofs. Empirical evaluations on models ranging from 1.1B to 13B parameters demonstrate that proof generation requires only 41 to 131 seconds, while verification time remains below 0.5 seconds for a sequence length of 512.

Abstract

Auditing the claimed outcomes of large language model (LLM) training is challenging when model weights and training data are private, while cryptographically proving the full training process is prohibitively expensive at Transformer scale. We present zkLLMPoT, a zero-knowledge framework that certifies auditor-defined properties of a trained checkpoint through forward evaluation rather than verification of its optimization trajectory. zkLLMPoT includes 2 phases: 1) The trainer fixes the architecture and the model weights are committed. Then the auditor selects challenge sequences, preventing the trainer from modifying the checkpoint in response to the audit data. 2) Then the trainer proves the objective value attained by the committed model on those sequences. This formulation makes the certification cost independent of the number of training iterations, without revealing model weights or requiring access to private training data. We build on sumcheck- and lookup-based arguments to certify Transformer computations, while supporting next-token loss and task-specific audit objectives. Across four model families, operator-level benchmarks yield proving times of 41-59 seconds for 1.1-1.5B-parameter models and 131 seconds at 13B for the covered operators, with verification below half a second at a sequence length of 512.

Problem

• Zero Knowledge Proof
• Large Language Models
• Training Auditing
• Privacy-preserving Verification

Innovation

• Zero-Knowledge Proof
• Large Language Models
• Forward Evaluation
• Sumcheck Protocol
• Model Auditing