PreGIP: Watermarking the Pretraining of Graph Neural Networks for Deep Intellectual Property Protection

2024-02-06

arXiv.org

AI Summary

To address the lack of intellectual property protection for pre-trained Graph Neural Networks (GNNs) and the limitations of existing watermarking methods—namely, their dependence on downstream tasks and insufficient robustness—this paper proposes the first task-agnostic watermarking framework operating directly at the pre-training stage. Methodologically, we design a watermark loss function constrained by embedding-space perturbations, introduce a fine-tuning-resistant watermark injection mechanism, and theoretically establish watermark unremovability, detectability, and embedding fidelity. Extensive experiments on multiple graph benchmark datasets demonstrate >99% watermark detection accuracy and <0.5% average degradation in downstream task performance—significantly outperforming state-of-the-art approaches. Our core contribution is the first watermarking scheme that embeds verifiable watermarks into GNN encoders *without requiring any downstream task*, achieving both strong robustness against removal attacks and minimal impact on model utility—thereby establishing a novel paradigm for copyright protection of pre-trained GNN models.

Technology Category

• Machine Learning: Graph-based Machine Learning
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: Language Grounding & Multi-modal NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Pretraining on Graph Neural Networks (GNNs) has shown great power in facilitating various downstream tasks. As pretraining generally requires huge amount of data and computational resources, the pretrained GNNs are high-value Intellectual Properties (IP) of the legitimate owner. However, adversaries may illegally copy and deploy the pretrained GNN models for their downstream tasks. Though initial efforts have been made to watermark GNN classifiers for IP protection, these methods require the target classification task for watermarking, and thus are not applicable to self-supervised pretraining of GNN models. Hence, in this work, we propose a novel framework named PreGIP to watermark the pretraining of GNN encoder for IP protection while maintain the high-quality of the embedding space. PreGIP incorporates a task-free watermarking loss to watermark the embedding space of pretrained GNN encoder. A finetuning-resistant watermark injection is further deployed. Theoretical analysis and extensive experiments show the effectiveness of {method} in IP protection and maintaining high-performance for downstream tasks.

Problem

• Protect pretrained GNNs from illegal copying
• Watermark self-supervised GNN pretraining without task dependency
• Maintain embedding quality while injecting robust watermarks

Innovation

• Watermarking pretrained GNN encoder for IP protection
• Task-free watermarking loss for embedding space
• Finetuning-resistant watermark injection technique