Breach in the Shield: Unveiling the Vulnerabilities of Large Language Models

2025-03-28

arXiv.org

AI Summary

This study identifies the root cause of LLMs’ and VLMs’ vulnerability to adversarial perturbations—namely, the heightened sensitivity of critical model parameters to input dimensions (e.g., token embeddings or pixel values). Method: We propose the first information-geometric framework for measuring invariance stability, unifying the assessment of parameter perturbation robustness and the localization of fragile regions in input space. Our approach integrates statistical sensitivity analysis, stability diagnostics in token embedding space, and perturbation-aware model merging optimization. Results: Evaluated on models ranging from 1.5B to 13B parameters, our method accurately pinpoints vulnerable parameters and image regions susceptible to adversarial attacks, while significantly enhancing the robustness and downstream task performance of merged models. The core contribution is the first theoretically rigorous yet practically interpretable quantification paradigm for multimodal model stability.

Technology Category

• Computer Vision: Adversarial Attacks & Robustness
• Machine Learning: Adversarial Learning & Robustness
• Natural Language Processing: Safety and Robustness

Application Category

• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large Language Models (LLMs) and Vision-Language Models (VLMs) have become essential to general artificial intelligence, exhibiting remarkable capabilities in task understanding and problem-solving. However, the real-world reliability of these models critically depends on their stability, which remains an underexplored area. Despite their widespread use, rigorous studies examining the stability of LLMs under various perturbations are still lacking. In this paper, we address this gap by proposing a novel stability measure for LLMs, inspired by statistical methods rooted in information geometry. Our measure possesses desirable invariance properties, making it well-suited for analyzing model sensitivity to both parameter and input perturbations. To assess the effectiveness of our approach, we conduct extensive experiments on models ranging in size from 1.5B to 13B parameters. Our results demonstrate the utility of our measure in identifying salient parameters and detecting vulnerable regions in input images or critical dimensions in token embeddings. Furthermore, leveraging our stability framework, we enhance model robustness during model merging, leading to improved performance.

Problem

• Identify vulnerable parameters in LLMs and VLMs
• Measure sensitivity using FI stability metric
• Improve model robustness by mitigating vulnerable parameters

Innovation

• Proposes FI stability measure from information geometry
• Identifies vulnerable parameters and input dimensions
• Mitigates influence during model merging