SecureGate: Learning When to Reveal PII Safely via Token-Gated Dual-Adapters for Federated LLMs

2026-02-13

AI Summary

This work addresses the challenge of balancing global generalization and local utility while preserving personally identifiable information (PII) in federated large language model (LLM) training. The authors propose SecureGate, a novel framework that introduces token-level gating combined with a dual-adapter LoRA architecture. During inference, SecureGate dynamically decides whether to activate the local adapter containing sensitive information, enabling fine-grained, on-demand PII disclosure without requiring retraining. Experimental results across multiple LLMs and real-world datasets demonstrate that SecureGate reduces the accuracy of PII inference attacks by up to 31.66× and decreases unauthorized extraction recall by 17.07×, while achieving 100% routing accuracy and incurring minimal communication and computational overhead.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Computer Vision: Large Vision Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Security and Privacy: Large-scale security measurements

Abstract

Federated learning (FL) enables collaborative training across organizational silos without sharing raw data, making it attractive for privacy-sensitive applications. With the rapid adoption of large language models (LLMs), federated fine-tuning of generative LLMs has gained attention as a way to leverage distributed data while preserving confidentiality. However, this setting introduces fundamental challenges: (i) privacy leakage of personally identifiable information (PII) due to LLM memorization, and (ii) a persistent tension between global generalization and local utility under heterogeneous data. Existing defenses, such as data sanitization and differential privacy, reduce leakage but often degrade downstream performance. We propose SecureGate, a privacy-aware federated fine-tuning framework for LLMs that provides fine-grained privacy control without sacrificing utility. SecureGate employs a dual-adapter LoRA architecture: a secure adapter that learns sanitized, globally shareable representations, and a revealing adapter that captures sensitive, organization-specific knowledge. A token-controlled gating module selectively activates these adapters at inference time, enabling controlled information disclosure without retraining. Extensive experiments across multiple LLMs and real-world datasets show that SecureGate improves task utility while substantially reducing PII leakage, achieving up to a 31.66X reduction in inference attack accuracy and a 17.07X reduction in extraction recall for unauthorized requests. Additionally, it maintains 100% routing reliability to the correct adapter and incurs only minimal computational and communication overhead.

Problem

• federated learning
• large language models
• PII leakage
• privacy-utility trade-off
• LLM memorization

Innovation

• token-gated dual-adapters
• federated LLM fine-tuning
• PII leakage prevention
• privacy-aware inference
• LoRA architecture