Scaling of Wireless Foundation Models via Representation Diversity and Multi-Branch Architectures

2026-10-03

AI Summary

This study addresses the unstable scaling benefits of wireless foundation models under data-limited regimes by proposing objective diversity as a novel scaling axis in place of parameter size. Methodologically, we design a multi-branch architecture with a shared backbone that integrates reconstruction, prediction, and contrastive learning objectives. This enables self-supervised pre-training on heterogeneous spectrogram and channel state information (CSI) data within a single-encoder budget while preserving complementary multi-objective information. Experimental results demonstrate that this orthogonal representation strategy consistently outperforms single-objective baselines across six downstream tasks, including communication-related applications. Furthermore, it significantly reduces parameter counts while validating the performance gains achieved through multi-objective fusion.

Abstract

Wireless foundation models learn representations from unlabeled radio signals for reuse across downstream tasks. Scaling model capacity is a common strategy for learning richer representations and improving downstream performance. However, its gains are less consistent in wireless self-supervised learning when pretraining data are limited. We investigate objective diversity as an alternative scaling axis: different self-supervised objectives emphasize different signal properties, and combining their representations can preserve more information to enable diverse tasks. We develop a fusion framework that combines frozen representations from independent encoders trained through reconstructive, predictive, and contrastive learning. This provides a reference for the benefits of diversity, but requires the maintenance of multiple encoders. To retain these benefits within the parameter budget of a standard single-objective encoder, we introduce a jointly trained multi-branch architecture with a shared trunk and objective-specific branches. We pretrain on a heterogeneous corpus of spectrogram and channel state information data and evaluate six downstream tasks that span communication, sensing, and positioning. At an equal output embedding dimension, fusion outperforms the evaluated single-objective encoders on all six tasks while using roughly one-third of their parameters. The multi-branch architecture retains much of the fusion benefit within a single-encoder parameter budget. Representation analyses indicate complementary contributions across objectives, with much of the added benefit retained in components orthogonal to the reconstruction representation subspace. These findings support objective diversity as an effective strategy for scaling wireless foundation models.

Problem

• Wireless Foundation Models
• Self-Supervised Learning
• Representation Diversity
• Model Scaling
• Multi-Branch Architectures

Innovation

• Wireless Foundation Models
• Representation Diversity
• Multi-Branch Architecture
• Self-Supervised Learning
• Objective Diversity