SafeCOMM: What about Safety Alignment in Fine-Tuned Telecom Large Language Models?

2025-05-29

AI Summary

This work identifies a pervasive security alignment degradation problem in fine-tuning large language models (LLMs) for telecommunications: even standard instruction tuning on structured domain data (e.g., 3GPP specifications) significantly impairs the model’s ability to refuse harmful or unethical queries. To address this, we propose SafeCOMM—the first security re-alignment framework tailored to telecom scenarios—comprising three novel techniques: security-aware instruction tuning (SafeInstruct), low-rank security adaptation (SafeLoRA), and security-guided model merging (SafeMERGE). Evaluated via red-teaming benchmarks and multi-paradigm safety enhancement—including secure continual pretraining, supervised fine-tuning, and LoRA-based merging—SafeCOMM achieves an average 76.3% reduction in harmful response rates across diverse telecom LLMs, with zero accuracy degradation on downstream tasks. This marks the first demonstration of simultaneous security strengthening and performance preservation in domain-specific LLM adaptation.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Safety and Robustness
• Application Domains: Security

Application Category

• Security and Privacy: Large-scale security measurements
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation

Abstract

Fine-tuning large language models (LLMs) for telecom tasks and datasets is a common practice to adapt general-purpose models to the telecom domain. However, little attention has been paid to how this process may compromise model safety. Recent research has shown that even benign fine-tuning can degrade the safety alignment of LLMs, causing them to respond to harmful or unethical user queries. In this paper, we investigate this issue for telecom-tuned LLMs using three representative datasets featured by the GenAINet initiative. We show that safety degradation persists even for structured and seemingly harmless datasets such as 3GPP standards and tabular records, indicating that telecom-specific data is not immune to safety erosion during fine-tuning. We further extend our analysis to publicly available Telecom LLMs trained via continual pre-training, revealing that safety alignment is often severely lacking, primarily due to the omission of safety-focused instruction tuning. To address these issues in both fine-tuned and pre-trained models, we conduct extensive experiments and evaluate three safety realignment defenses (SafeInstruct, SafeLoRA, and SafeMERGE) using established red-teaming benchmarks. The results show that, across all settings, the proposed defenses can effectively restore safety after harmful degradation without compromising downstream task performance, leading to Safe teleCOMMunication (SafeCOMM) models. In a nutshell, our work serves as a diagnostic study and practical guide for safety realignment in telecom-tuned LLMs, and emphasizes the importance of safety-aware instruction and fine-tuning for real-world deployments of Telecom LLMs.

Problem

• Investigates safety degradation in telecom-tuned LLMs during fine-tuning
• Assesses lack of safety alignment in publicly available Telecom LLMs
• Evaluates defenses to restore safety without compromising task performance

Innovation

• Investigates safety degradation in telecom-tuned LLMs
• Evaluates three safety realignment defenses effectively
• Emphasizes safety-aware instruction for Telecom LLMs