Neural-Enhanced Rate Adaptation and Computation Distribution for Emerging mmWave Multi-User 3D Video Streaming Systems

2025-05-19

AI Summary

This work addresses the joint optimization of communication and computation resources for 360° video streaming in millimeter-wave multi-user VR systems. Method: We propose MTRC, a deep reinforcement learning framework, along with two novel neural cascade variants—R1C2 and C1R2—that explicitly model multi-task–multi-user action coupling without requiring prior environmental knowledge. The framework jointly optimizes bitrate adaptation and edge computing task allocation under constraints on rebuffering latency and quality fluctuation to maximize user Quality of Experience (QoE). Results: Experiments demonstrate that C1R2 achieves state-of-the-art performance: it improves viewport PSNR by 5.21–6.06 dB, reduces stalling duration by 2.18–2.70×, and suppresses quality fluctuation by 4.14–4.50 dB compared to existing methods—significantly enhancing system robustness and user experience.

Technology Category

• Computer Vision: Learning & Optimization for CV
• Machine Learning: Hardware-aware ML
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Virtualization and resource management in Web systems and infrastructures
• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: LLM based quality controls for crowd work

Abstract

We investigate multitask edge-user communication-computation resource allocation for $360^circ$ video streaming in an edge-computing enabled millimeter wave (mmWave) multi-user virtual reality system. To balance the communication-computation trade-offs that arise herein, we formulate a video quality maximization problem that integrates interdependent multitask/multi-user action spaces and rebuffering time/quality variation constraints. We formulate a deep reinforcement learning framework for underline{m}ulti-underline{t}ask underline{r}ate adaptation and underline{c}omputation distribution (MTRC) to solve the problem of interest. Our solution does not rely on a priori knowledge about the environment and uses only prior video streaming statistics (e.g., throughput, decoding time, and transmission delay), and content information, to adjust the assigned video bitrates and computation distribution, as it observes the induced streaming performance online. Moreover, to capture the task interdependence in the environment, we leverage neural network cascades to extend our MTRC method to two novel variants denoted as R1C2 and C1R2. We train all three methods with real-world mmWave network traces and $360^circ$ video datasets to evaluate their performance in terms of expected quality of experience (QoE), viewport peak signal-to-noise ratio (PSNR), rebuffering time, and quality variation. We outperform state-of-the-art rate adaptation algorithms, with C1R2 showing best results and achieving $5.21-6.06$ dB PSNR gains, $2.18-2.70$x rebuffering time reduction, and $4.14-4.50$ dB quality variation reduction.

Problem

• Optimize communication-computation resource allocation for mmWave multi-user 3D video streaming
• Maximize video quality with multitask constraints and rebuffering limits
• Develop deep reinforcement learning for adaptive bitrate and computation distribution

Innovation

• Deep reinforcement learning for rate adaptation
• Neural network cascades for task interdependence
• Online adjustment using streaming statistics