Privacy-Preserving Semantic Segmentation without Key Management

2026-04-16

AI Summary

This work addresses the challenge of image privacy in semantic segmentation by proposing an end-to-end encrypted training and inference framework that eliminates the need for centralized key management. The method enables model creators and clients to encrypt input images using locally generated keys that are independent across users and images, and integrates the encryption strategy directly into the training process to mitigate performance degradation. Built upon the SETR architecture, this approach achieves, for the first time, privacy-preserving semantic segmentation with per-client, per-image unique keys—without requiring key distribution or sharing. Experimental results on the Cityscapes dataset demonstrate that the proposed method maintains high segmentation accuracy while rigorously preserving image privacy.

Technology Category

• Computer Vision: Segmentation
• Machine Learning: Privacy
• Philosophy and Ethics of AI: Privacy & Security

Application Category

• Security and Privacy: Applications of cryptography
• Semantics and Knowledge: Provenance, trust, security and privacy, and ethical issues in managing semantic data
• Responsible Web: Data and user privacy-enhancing technologies for the Web

Abstract

This paper proposes a novel privacy-preserving semantic segmentation method that can use independent keys for each client and image. In the proposed method, the model creator and each client encrypt images using locally generated keys, and model training and inference are conducted on the encrypted images. To mitigate performance degradation, an image encryption method is applied to model training in addition to the generation of test images. In experiments, the effectiveness of the proposed method is confirmed on the Cityscapes dataset under the use of a vision transformer-based model, called SETR.

Problem

• privacy-preserving
• semantic segmentation
• key management
• image encryption
• federated learning

Innovation

• privacy-preserving
• semantic segmentation
• keyless encryption
• vision transformer
• encrypted domain learning