WALoMA: A Multitask Wireless Foundation Model via Adaptive Low-Rank Masked Autoencoders

2026-07-28

AI Summary

This work addresses the challenges of poor generalization and scarce labeled data in task-specific deep learning models for 6G wireless physical layers by proposing a unified multitask wireless foundation model. Leveraging channel state information as a universal modality, the model employs a masked autoencoder to learn transferable representations through self-supervised pretraining on unlabeled data. It incorporates two-dimensional positional encoding to preserve the spatial-frequency structure across antennas and subcarriers and utilizes low-rank adaptation (LoRA) for efficient fine-tuning. Requiring updates to only 14.68% of parameters on average, the model achieves a comprehensive downstream performance of 87.80% across five tasks, substantially outperforming baseline models at 59.90%, thereby demonstrating superior generalization and effectiveness under extremely limited labeled data.

Abstract

This paper proposes a multitask wireless foundation model via adaptive low-rank masked autoencoders (WALoMA), a unified multi-task foundation model for sixth-generation (6G) wireless physical layer architectures, to address the limitations of specialized, task-specific deep learning models and the practical challenge of scarce labeled wireless datasets. By leveraging concepts inspired by foundation models, the proposed framework adopts a masked autoencoder (MAE) paradigm to learn from unlabeled channel data, to significantly reduce reliance on extensive annotations. The model treats wireless channel state information (CSI) as a universal modality and learns transferable representations through self-supervised channel reconstruction. Key architectural novelties include the use of 2D positional encoding (PE) to explicitly preserve the spatial-frequency relationships between antennas and subcarriers, and low-rank adaptation (LoRA) for parameter-efficient fine-tuning. The framework's efficacy is demonstrated across five downstream tasks, achieving individual scores of 96.47\% for LoS/NLoS classification, 80.45\% for beam prediction, 85.78\% for channel interpolation, 99.12\% for channel estimation, and 77.18\% for channel charting. Consequently, numerical results show that the proposed model achieves a composite score of 87.80\%, significantly outperforming the 59.90\% achieved by the large wireless model (LWM) baseline while training an average of only 14.68\% of total parameters, and maintaining strong performance even under extremely limited labeled data conditions.

Problem

• wireless foundation model
• multitask learning
• labeled data scarcity
• channel state information
• 6G physical layer

Innovation

• masked autoencoder
• low-rank adaptation
• wireless foundation model
• channel state information
• self-supervised learning