Real-Time Semantic Segmentation of Aerial Images Using an Embedded U-Net: A Comparison of CPU, GPU, and FPGA Workflows

2025-03-07

AI Summary

Real-time semantic segmentation of aerial imagery on resource-constrained embedded platforms demands lightweight models and efficient hardware-aware deployment. Method: We propose a lightweight U-Net architecture tailored for embedded aviation applications, reducing parameters and MACs by 16× while preserving accuracy. We conduct the first systematic benchmark across three commercial hardware platforms—CPU, GPU, and FPGA—and five deployment toolchains: TVM, FINN, Vitis AI, and TensorFlow GPU/cuDNN—evaluating latency, power consumption, memory footprint, energy efficiency, and FPGA resource utilization. Contribution/Results: Our evaluation reveals that the FPGA–Vitis AI combination achieves optimal real-time performance (<12 ms inference latency) and energy efficiency—3.8× higher than GPU—while reducing power consumption by 67%. This work highlights the critical role of hardware–compiler co-optimization for edge-intelligent remote sensing and establishes a reproducible heterogeneous computing benchmark and deployment paradigm for real-time aerial image segmentation.

Technology Category

Machine Learning: Hardware-aware MLComputer Vision: SegmentationPlanning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applicationsSearch and Retrieval-Augmented AI: Web evaluation methodologies and metricsSemantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

This study introduces a lightweight U-Net model optimized for real-time semantic segmentation of aerial images, targeting the efficient utilization of Commercial Off-The-Shelf (COTS) embedded computing platforms. We maintain the accuracy of the U-Net on a real-world dataset while significantly reducing the model's parameters and Multiply-Accumulate (MAC) operations by a factor of 16. Our comprehensive analysis covers three hardware platforms (CPU, GPU, and FPGA) and five different toolchains (TVM, FINN, Vitis AI, TensorFlow GPU, and cuDNN), assessing each on metrics such as latency, power consumption, memory footprint, energy efficiency, and FPGA resource usage. The results highlight the trade-offs between these platforms and toolchains, with a particular focus on the practical deployment challenges in real-world applications. Our findings demonstrate that while the FPGA with Vitis AI emerges as the superior choice due to its performance, energy efficiency, and maturity, it requires specialized hardware knowledge, emphasizing the need for a balanced approach in selecting embedded computing solutions for semantic segmentation tasks

Problem

• Optimize U-Net for real-time aerial image segmentation.
• Compare CPU, GPU, FPGA workflows for efficiency metrics.
• Assess deployment challenges in embedded computing platforms.

Innovation

• Lightweight U-Net for real-time aerial image segmentation
• Comparison of CPU, GPU, and FPGA workflows
• FPGA with Vitis AI excels in performance and efficiency