Rethinking Streaming-Perception Evaluation on Heterogeneous Edge Platforms

2026-10-04

AI Summary

This study addresses the challenges of erroneous deployment decisions caused by isolated evaluation and degraded perception performance due to resource contention on heterogeneous edge platforms. We construct an end-to-end multi-camera streaming perception pipeline on a GPU-NPU platform. By simulating co-located vision-language model workloads, we reveal ranking discrepancies between isolated evaluations and real-world deployments, and propose a comprehensive evaluation paradigm integrating contention profiling, deadline miss rate, and worst-stream metrics. Experimental results demonstrate that under high contention, an all-NPU deployment improves the worst-stream sAP by 5.2×, establishing an optimized placement strategy grounded in real-time constraints.

Abstract

Multi-camera streaming perception is increasingly deployed on heterogeneous edge platforms shared with co-resident workloads, yet accelerator placement is often evaluated using isolated single-stream experiments and mean streaming average precision (sAP). Using two end-to-end pipelines on a single GPU--NPU platform, we show that isolated evaluation can mis-rank deployment-time placement. Although the GPU pipeline is preferred in isolation, GPU-localized contention introduces deadline misses that make detections stale and can reverse the preferred placement before full GPU saturation. The NPU pipeline is less accurate than the GPU pipeline on small and medium objects in isolation, but nearly matches it on large objects. The largest absolute sAP losses in our latency and contention experiments occur for large objects. In our four-stream experiments, the preferred placement depends on which path becomes stale, and increasing GPU-side contention shifts the best placement from All-GPU to All-NPU. Under a GPU-saturating vision--language co-tenant, All-NPU achieves $5.2\times$ the worst-stream sAP of All-GPU. Because mean sAP can hide severe single-stream degradation, evaluation should report contention sweeps, deadline-miss rates on both paths, and worst-stream sAP alongside mean sAP.

Problem

• streaming perception
• heterogeneous edge platforms
• accelerator placement evaluation
• resource contention
• mean streaming average precision

Innovation

• Streaming Perception
• Heterogeneous Edge Platforms
• Accelerator Placement
• Resource Contention
• Worst-stream sAP