StraightDP: Geometry-Aware Differential Privacy for Rectified-Flow Transformers

2026-07-31

AI Summary

This work addresses the significant utility degradation commonly observed in text-conditional generative models under stringent privacy constraints due to differentially private training. The authors propose a two-stage training approach leveraging the geometric properties of rectified flows: first, a small privacy budget is allocated to release class-conditional moments for model initialization or sampling injection; the remaining budget is then used for data-side optimization via DP-SGD. This method exploits, for the first time, the structural heterogeneity along rectified flow trajectories to decouple generic statistics from sample-specific information, enabling efficient privacy budget allocation and seamless transfer to pretrained diffusion models. Experiments show that on MNIST with ε=1, using only the released moments achieves 0.76 accuracy and FID 237, while the full pipeline attains 0.81 accuracy and FID 56, outperforming DP-LoRA on a frozen SD3-medium model at a lower privacy cost.

Abstract

Differentially private (DP) training of text-conditioned generative models suffers a utility cliff at strong privacy. We revisit this problem through the geometry of rectified flows: along the straight interpolation between noise and data, the Bayes-optimal velocity is governed to leading order at the noise end by a few class-conditional moments, and increasingly sample-specific structure matters toward the data end. StraightDP exploits this heterogeneity end to end. A small budget share releases whitened class-conditional moments once, to be distilled into the weights or injected at sampling time. The rest is spent by pre-declared DP-SGD toward the data end, beyond the moments' reach. At $\varepsilon=1$ on MNIST, the released moments alone already attain $0.76$ downstream accuracy with prototype-like samples and an FID of $237$, and uniform DP-SGD attains $0.21$. The pipeline built on the release reaches $0.81$ accuracy at FID $56$ in a public latent space. Constraining per-token stream norms of the multimodal backbone leaves the pretraining loss unchanged yet improves downstream accuracy in the extreme-noise pixel-space regime, and its accuracy effect becomes monotonically more favorable as privacy strengthens. The released moments also port to frozen SD3-medium, where sampling-time injection beats DP-LoRA training at a fraction of the budget.

Problem

• Differential Privacy
• Generative Models
• Utility Cliff
• Rectified Flows
• Text-Conditioned Generation

Innovation

• Rectified Flow
• Differential Privacy
• Class-Conditional Moments
• DP-SGD
• Geometry-Aware Privacy