Mining Limited Data Sufficiently: A BERT-inspired Approach for CSI Time Series Application in Wireless Communication and Sensing

2024-12-09

arXiv.org

AI Summary

To address the challenges of scarce CSI time-series data, high annotation costs, heterogeneous formats, and short coherence times—limiting prediction and classification performance in integrated sensing and communication (ISAC) systems—this paper proposes CSI-BERT2, the first pretraining-finetuning framework tailored for CSI time-series modeling. Methodologically, it introduces an Adaptive Reweighting Layer (ARL) and a subcarrier-time dual-sensitive MLP architecture to overcome permutation invariance inherent in CSI sequences, and incorporates Masked Prediction Fine-tuning (MPM) to enhance few-shot generalization. Built upon the BERT architecture, CSI-BERT2 unifies self-supervised pretraining with joint time-frequency representation learning. Evaluated on multi-task CSI prediction and fine-grained activity classification, it achieves state-of-the-art (SOTA) performance, notably improving accuracy by 12.7%–23.4% in ultra-low-data regimes (<1k labeled samples).

Technology Category

Machine Learning: Calibration & Uncertainty QuantificationCognitive Modeling & Cognitive Systems: Adaptive BehaviorIntelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applicationsSearch and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for rankingWeb Mining and Content Analysis: Large pretrained models with web data

Abstract

Channel State Information (CSI) is the cornerstone in both wireless communication and sensing systems. In wireless communication systems, CSI provides essential insights into channel conditions, enabling system optimizations like channel compensation and dynamic resource allocation. However, the high computational complexity of CSI estimation algorithms necessitates the development of fast deep learning methods for CSI prediction. In wireless sensing systems, CSI can be leveraged to infer environmental changes, facilitating various functions, including gesture recognition and people identification. Deep learning methods have demonstrated significant advantages over model-based approaches in these fine-grained CSI classification tasks, particularly when classes vary across different scenarios. However, a major challenge in training deep learning networks for wireless systems is the limited availability of data, further complicated by the diverse formats of many public datasets, which hinder integration. Additionally, collecting CSI data can be resource-intensive, requiring considerable time and manpower. To address these challenges, we propose CSI-BERT2 for CSI prediction and classification tasks, effectively utilizing limited data through a pre-training and fine-tuning approach. Building on CSI-BERT1, we enhance the model architecture by introducing an Adaptive Re-Weighting Layer (ARL) and a Multi-Layer Perceptron (MLP) to better capture sub-carrier and timestamp information, effectively addressing the permutation-invariance problem. Furthermore, we propose a Mask Prediction Model (MPM) fine-tuning method to improve the model's adaptability for CSI prediction tasks. Experimental results demonstrate that CSI-BERT2 achieves state-of-the-art performance across all tasks.

Problem

• Efficient CSI prediction in high-mobility wireless communication
• Improved CSI classification with scarce or lost data
• Unified framework for wireless sensing and communication tasks

Innovation

• Two-stage training with MLM and fine-tuning
• Mask prediction model for CSI prediction
• ARL and MLP for enhanced representation