UPAQ: A Framework for Real-Time and Energy-Efficient 3D Object Detection in Autonomous Vehicles

2025-01-08

AI Summary

To address the poor real-time performance, high power consumption, and stringent resource constraints of multimodal 3D object detection models on autonomous driving embedded platforms, this work proposes the first end-to-end compression framework integrating semi-structured pruning with mixed-precision quantization for LiDAR–image fusion detection models (PointPillars/SMOKE). We optimize the point cloud–image feature fusion architecture and implement hardware–software co-design deployment on the Jetson Orin Nano. Under zero accuracy degradation, our method achieves up to 5.62× parameter reduction, 1.97× inference speedup, and 2.07× energy savings. The approach significantly improves the energy efficiency and practical deployability of multimodal 3D detection on edge devices, providing a viable technical pathway for real-time perception in resource-constrained scenarios.

Technology Category

• Machine Learning: Learning on the Edge & Model Compression
• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

To enhance perception in autonomous vehicles (AVs), recent efforts are concentrating on 3D object detectors, which deliver more comprehensive predictions than traditional 2D object detectors, at the cost of increased memory footprint and computational resource usage. We present a novel framework called UPAQ, which leverages semi-structured pattern pruning and quantization to improve the efficiency of LiDAR point-cloud and camera-based 3D object detectors on resource-constrained embedded AV platforms. Experimental results on the Jetson Orin Nano embedded platform indicate that UPAQ achieves up to 5.62x and 5.13x model compression rates, up to 1.97x and 1.86x boost in inference speed, and up to 2.07x and 1.87x reduction in energy consumption compared to state-of-the-art model compression frameworks, on the Pointpillar and SMOKE models respectively.

Problem

• Autonomous Vehicles
• 3D Object Detection
• Resource Efficiency

Innovation

• 3D Object Detection
• Efficiency Improvement
• Resource Optimization