Development of Vision-Language Model-based GNSS Spoofing Detection for Autonomous Vehicle Navigation

2026-07-26

AI Summary

This work addresses the vulnerability of autonomous driving systems to GNSS spoofing attacks, which pose significant safety risks. The authors propose the first GNSS spoofing detection framework based on a vision-language model (VLM), fusing front-view images with vehicle dynamic sensor data. By employing a three-stage fine-tuning strategy, the method aligns GNSS-derived trajectories with multimodal predicted trajectories in a shared semantic space to identify inconsistencies indicative of spoofing. Key contributions include the novel integration of VLMs into this domain with a dedicated multimodal semantic alignment mechanism, an adaptive inference strategy that reduces VLM invocation overhead by 86% (achieving 65–73 ms per inference window), and the creation of a real-world, cross-regional road dataset along with a sophisticated spoofing attack model. Experiments demonstrate an F1 score of 94%–95%, 100% detection accuracy for wrong-turn and stop attacks, and 88%–93% accuracy against overshoot attacks.

Abstract

Autonomous vehicles (AVs) depend on Global Navigation Satellite Systems (GNSS) for localization and navigation, making them vulnerable to spoofing attacks that can covertly redirect vehicles or induce unsafe maneuvers. In this paper, we develop the first Vision-Language Model (VLM)-based framework for GNSS spoofing detection for autonomous vehicles by fusing front-camera visual data with in-vehicle sensor readings (e.g., speed, acceleration, yaw rate) against GNSS-derived maneuvers. Our approach introduces a three-stage fine-tuning process that first grounds visual cues, and then calibrates sensor data within a shared semantic space to detect discrepancies between predicted and GNSS-derived maneuvers across three attack scenarios. We also generated an independent real-world dataset by driving an instrumented vehicle on public roads in Tuscaloosa, Alabama, equipped with time-synchronized GNSS, IMU, and camera logs to validate cross-regional generalization of our fine-tuned model on unseen data from training data. On this dataset, we then generated intelligent spoofing attacks, including trajectory mirroring with road-network snapping for wrong-turn attacks, position freezing for overshoot scenarios, and drift generation for stop attacks. On this validation dataset, the zero-shot VLMs baseline F1-score ranges from 23% to 32%, whereas our fine-tuned model achieves an F1-score ranging from 94% to 95%. Results show that our VLM-based approach correctly classified every wrong-turn and stop attacks, and attains 88%-93% accuracy for overshoot attacks. Furthermore, we introduce an adaptive inference policy that reduces VLM invocations to 14% (~86% computational reduction) and yields 65ms-73ms per 4s window. These results point to a practical, on-road layer of defense that complements signal-level integrity checks with the use of VLMs.

Problem

• GNSS spoofing detection
• autonomous vehicles
• vision-language model
• navigation security
• spoofing attacks

Innovation

• Vision-Language Model
• GNSS Spoofing Detection
• Multimodal Sensor Fusion
• Adaptive Inference
• Autonomous Vehicle Navigation