Semantic-Aware Joint Source-Channel Optimization for Encoder-Agnostic Digital Video Communication

2026-09-30

AI Summary

This study addresses the challenge that semantic video communication relies on heavy codecs, hindering deployment in resource-constrained scenarios. To this end, it proposes an encoder-agnostic, lightweight, plug-and-play transmission framework. The method quantifies inter-frame semantic importance by combining cosine similarity with a shifted window mechanism and dynamically adjusts source-channel coding parameters according to channel conditions via a multi-agent proximal policy optimization algorithm, enabling adaptation to arbitrary encoders without retraining. Experimental results demonstrate that integrating this scheme with H.265 and DCVC-RT achieves Bjøntegaard Delta rate savings of 34.86% and 18.01%, respectively, alongside a PSNR improvement of 1.448 dB and an LPIPS reduction of 0.033.

Abstract

Video semantic communication has attracted increasing attention as a promising approach to improving video transmission efficiency. However, most existing approaches rely on computationally intensive deep learning-based video encoders and decoders, which hinders their deployment in resource-constrained scenarios. To address this issue, we propose a lightweight semantic-aware joint source-channel optimization (SAJSCO) scheme that can be integrated into existing digital video communication systems as a plug-in module. Specifically, we develop a video communication system model in which the transmitter jointly optimizes source and channel coding parameters based on the inter-frame semantic importance of the input video and estimated channel state information. On this basis, we formulate an optimization problem that maximizes semantic importance weighted video reconstruction quality under a maximum bitrate constraint. To solve it, we first quantify inter-frame semantic importance using a cosine similarity-based metric with a shifted window mechanism. We then develop a multi-actor proximal policy optimization (MPPO) algorithm to solve the formulated problem by jointly adapting the source compression rate and channel coding rate. The learned policy can be directly applied to different video encoders without encoder-specific retraining or fine-tuning. SAJSCO achieves Bjøntegaard Delta rate reductions of 34.86\% and 18.01\% when integrated with H.265, a conventional video encoder, and DCVC-RT, a deep learning-based video encoder. Over-the-air experiments on a hardware testbed further demonstrate a PSNR gain of up to 1.448 dB with H.265 and an LPIPS reduction of up to 0.033 with DCVC-RT compared with the respective best-performing fixed-parameter baselines.

Problem

• Semantic Communication
• Video Transmission
• Joint Source-Channel Optimization
• Resource-Constrained
• Encoder-Agnostic

Innovation

• Semantic-Aware Joint Source-Channel Optimization
• Encoder-Agnostic
• Multi-Actor Proximal Policy Optimization
• Inter-frame Semantic Importance
• Plug-in Module