Beyond VMAF: Towards Application-Specific Metrics for Teleoperation Video

2026-05-13

AI Summary

This study addresses the limitations of general-purpose video quality metrics—such as VMAF—in accurately capturing human perception of task-relevant regions in safety-critical scenarios like remote driving. To bridge this gap, the authors introduce, for the first time, human subjective quality ratings from the teleoperation domain to perform domain-adaptive retraining of VMAF, yielding a specialized video quality assessment model tailored for remote driving applications. Evaluated on the Zenseact dataset using RMSE and MAD as performance indicators, the retrained model demonstrates significantly improved alignment with human perception: RMSE decreases by 15% (from 10.36 to 8.83) and MAD by 27% (from 8.71 to 6.38), effectively mitigating the blind spots of generic metrics in assessing distortions within critical visual regions.

Abstract

Automated driving has made remarkable progress, yet situations still arise where human intervention is necessary. Teleoperation provides a scalable solution to address such cases, enabling remote operators to support vehicles without being physically present. In this context, video transmission forms the operator's primary source of situational awareness, making video quality a decisive factor for both safety and task performance. In an online study, participants rated compressed video sequences from the Zenseact Dataset and provided subjective quality ratings. These ratings were then used to retrain the Video Multi-Method Assessment Fusion (VMAF) model, yielding an adapted variant tailored to teleoperation. The retrained model demonstrated improved alignment with human ratings compared to the original 4K VMAF. In particular, RMSE decreased from 10.36 to 8.83, and MAD from 8.71 to 6.38, corresponding to improvements of 15% and 27%, respectively. These results highlight that incorporating domain-specific data can enhance the predictive power of established quality metrics in safety-critical applications. At the same time, Outlier cases emerged in which videos received high objective scores despite noticeable degradations in regions critical for the driving task.

Problem

• teleoperation
• video quality assessment
• VMAF
• subjective quality
• autonomous driving

Innovation

• teleoperation
• application-specific video quality
• VMAF adaptation
• subjective quality assessment
• safety-critical systems