Adversarial Vulnerabilities in Large Language Models for Time Series Forecasting

2024-12-11

AI Summary

This work presents the first systematic investigation of significant adversarial vulnerabilities in large language models (LLMs) for time-series forecasting. Addressing limitations of existing gradient-dependent attacks—such as poor generalizability and reliance on model internals—we propose the first targeted adversarial attack framework specifically designed for LLM-based time-series prediction. Our approach integrates gradient-free optimization with black-box search to efficiently generate minimal, input-agnostic perturbations. Extensive experiments across diverse architectures—including TimeGPT, LLM-Time, GPT-3.5/4, LLaMA, and Mistral—demonstrate substantial degradation in forecasting accuracy across multiple real-world time-series benchmarks. The induced perturbations consistently outperform random noise in attack efficacy. These results empirically establish widespread robustness deficiencies in current LLM-based forecasting systems, providing both critical evidence and a standardized benchmark for developing future defense mechanisms.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Computer Vision: Adversarial Attacks & Robustness
• Natural Language Processing: (Large) Language Models

Application Category

• User Modeling, Personalization and Recommendation: Large Language Models (LLM) for user modeling and recommendation
• Search and Retrieval-Augmented AI: Search Tool Learning with LLM: Teaching LLMs to invoke search and make use of retrieved information
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs

Abstract

Large Language Models (LLMs) have recently demonstrated significant potential in the field of time series forecasting, offering impressive capabilities in handling complex temporal data. However, their robustness and reliability in real-world applications remain under-explored, particularly concerning their susceptibility to adversarial attacks. In this paper, we introduce a targeted adversarial attack framework for LLM-based time series forecasting. By employing both gradient-free and black-box optimization methods, we generate minimal yet highly effective perturbations that significantly degrade the forecasting accuracy across multiple datasets and LLM architectures. Our experiments, which include models like TimeGPT and LLM-Time with GPT-3.5, GPT-4, LLaMa, and Mistral, show that adversarial attacks lead to much more severe performance degradation than random noise, and demonstrate the broad effectiveness of our attacks across different LLMs. The results underscore the critical vulnerabilities of LLMs in time series forecasting, highlighting the need for robust defense mechanisms to ensure their reliable deployment in practical applications.

Problem

• Large Language Models
• Time Series Prediction
• Adversarial Attacks

Innovation

• Adversarial Attacks
• Time Series Prediction
• Large Language Models (LLMs)