SEG-JPEG: Simple Visual Semantic Communications for Remote Operation of Automated Vehicles over Unreliable Wireless Networks

2026-02-16

AI Summary

This work proposes a lightweight visual semantic communication scheme to address the challenges of high bandwidth demand and data loss in remote operation of autonomous vehicles over unreliable wireless networks. The approach overlays semantically segmented road users as color highlights onto low-resolution grayscale images and transmits the composite frames using JPEG compression. By integrating semantic segmentation with conventional image compression—a novel combination for teleoperation—this method achieves end-to-end latency below 200 ms at bandwidths under 500 kbit/s, reducing data rates by approximately 50% while preserving critical visual information. Experimental results demonstrate significantly enhanced operational reliability and situational awareness in poor network conditions, validating the feasibility of large-scale deployment over real-world 4G networks.

Technology Category

• Computer Vision: Segmentation
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems
• Multiagent Systems: Agent Communication

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Search and Retrieval-Augmented AI: Efficiency and scalability of Web search engines
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Remote Operation is touted as being key to the rapid deployment of automated vehicles. Streaming imagery to control connected vehicles remotely currently requires a reliable, high throughput network connection, which can be limited in real-world remote operation deployments relying on public network infrastructure. This paper investigates how the application of computer vision assisted semantic communication can be used to circumvent data loss and corruption associated with traditional image compression techniques. By encoding the segmentations of detected road users into colour coded highlights within low resolution greyscale imagery, the required data rate can be reduced by 50 \% compared with conventional techniques, while maintaining visual clarity. This enables a median glass-to-glass latency of below 200ms even when the network data rate is below 500kbit/s, while clearly outlining salient road users to enhance situational awareness of the remote operator. The approach is demonstrated in an area of variable 4G mobile connectivity using an automated last-mile delivery vehicle. With this technique, the results indicate that large-scale deployment of remotely operated automated vehicles could be possible even on the often constrained public 4G/5G mobile network, providing the potential to expedite the nationwide roll-out of automated vehicles.

Problem

• remote operation
• automated vehicles
• unreliable wireless networks
• image compression
• visual semantic communication

Innovation

• semantic communication
• computer vision
• image compression
• remote operation
• automated vehicles