FRESHLATENT: Channel-Aware Latent Adaptation for Resource-Constrained Embodied VLM Perception

2026-09-24

AI Summary

This study addresses feature distortion caused by wireless channel interference in UAV-based VLM perception and the prohibitive computational overhead of conventional robust codecs. To this end, we propose FreshLatent, a lightweight channel-aware latent adapter. This method trains a power-normalized codec under a frozen VLM to mitigate wireless noise and introduces the first deployment framework linking channel quality and communication budgets to task conditions, enabling efficient and robust perception under resource constraints. Experimental results demonstrate that FreshLatent improves gIoU by over 20 points at low signal-to-noise ratios while reducing parameter count by 37× and latency by nearly 10×. By recovering most robustness with minimal parametric cost, it significantly extends the effective operating range for UAV perception systems.

Abstract

Mission-critical UAVs increasingly rely on split vision-language model (VLM) perception under tight onboard-resource and wireless-communication constraints. However, corruption of transmitted intermediate features creates a deployment mismatch for clean-trained split interfaces, while stronger channel-aware codecs can impose substantial onboard cost. We present FreshLatent, a lightweight channel-aware latent adapter that trains a power-normalized encoder-decoder through wireless corruption while keeping the surrounding VLM frozen. We formulate deployment around a mission-conditioned perception requirement and embedded interface cost, linking channel quality and communication budget to the operating conditions under which perception remains usable. At 0 dB and the tightest communication budget, FreshLatent improves gIoU and cIoU over clean split compression by 20.79 and 20.87 points, respectively. At the most adverse evaluated SNR (0 dB), across all three communication budgets, FreshLatent recovers 63.5-69.1% of the gIoU improvement achieved by a much heavier, range-trained feature-JSCC codec. On an NVIDIA Jetson AGX Xavier in 10-W mode, FreshLatent uses 37-40x fewer encoder parameters, 7.7-9.9x lower edge-interface latency, and 8.8-10.0x lower edge-interface energy than the heavier codec. Together, these results show that lightweight channel-aware adaptation can recover a substantial fraction of the robustness of a much larger communication interface while broadening quality-valid operation under constrained wireless conditions.

Problem

• Embodied VLM
• Split perception
• Wireless channel corruption
• Resource-constrained UAV
• Deployment mismatch

Innovation

• Channel-Aware Latent Adaptation
• Split Vision-Language Model
• Resource-Constrained UAV
• Lightweight Codec
• Edge Deployment