RF-GPT: Teaching AI to See the Wireless World

2026-02-16

AI Summary

This work addresses the gap between wireless signal perception and high-level semantic reasoning by proposing the first radio frequency (RF) language model that natively integrates RF signals into a multimodal large language model framework. By converting in-phase/quadrature (IQ) waveforms into time-frequency spectrograms, the approach leverages a pretrained vision encoder to extract meaningful features, which are then injected into the language decoder as RF tokens, enabling end-to-end trainable RF semantic understanding without requiring manual annotations. The model demonstrates superior performance over general-purpose vision-language models across multiple tasks—including wideband modulation classification, wireless technology identification, and WLAN user counting—thereby validating its effectiveness and strong generalization capability in RF-aware multimodal reasoning.

Technology Category

• Machine Learning: Large Multimodal Models (LMMs)
• Natural Language Processing: Language Grounding & Multi-modal NLP
• Computer Vision: Multi-modal Vision

Application Category

• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Large language models (LLMs) and multimodal models have become powerful general-purpose reasoning systems. However, radio-frequency (RF) signals, which underpin wireless systems, are still not natively supported by these models. Existing LLM-based approaches for telecom focus mainly on text and structured data, while conventional RF deep-learning models are built separately for specific signal-processing tasks, highlighting a clear gap between RF perception and high-level reasoning. To bridge this gap, we introduce RF-GPT, a radio-frequency language model (RFLM) that utilizes the visual encoders of multimodal LLMs to process and understand RF spectrograms. In this framework, complex in-phase/quadrature (IQ) waveforms are mapped to time-frequency spectrograms and then passed to pretrained visual encoders. The resulting representations are injected as RF tokens into a decoder-only LLM, which generates RF-grounded answers, explanations, and structured outputs. To train RF-GPT, we perform supervised instruction fine-tuning of a pretrained multimodal LLM using a fully synthetic RF corpus. Standards-compliant waveform generators produce wideband scenes for six wireless technologies, from which we derive time-frequency spectrograms, exact configuration metadata, and dense captions. A text-only LLM then converts these captions into RF-grounded instruction-answer pairs, yielding roughly 12,000 RF scenes and 0.625 million instruction examples without any manual labeling. Across benchmarks for wideband modulation classification, overlap analysis, wireless-technology recognition, WLAN user counting, and 5G NR information extraction, RF-GPT achieves strong multi-task performance, whereas general-purpose VLMs with no RF grounding largely fail.

Problem

• radio-frequency signals
• large language models
• multimodal models
• RF perception
• wireless systems

Innovation

• RF-GPT
• radio-frequency language model
• multimodal LLM
• spectrogram encoding
• synthetic RF dataset