RoadFed: A Multimodal Federated Learning System for Improving Road Safety

2025-02-14

AI Summary

To address the limitations of single-modality perception, high computational/communication overhead, and insufficient privacy protection for high-dimensional multimodal data in Cooperative Intelligent Transport Systems (C-ITS) road hazard detection, this paper proposes an edge-end collaborative multimodal federated learning framework. We introduce the first road-safety-oriented multimodal federated architecture, design a communication-efficient federated optimization algorithm, and develop a low-error local differential privacy mechanism tailored to heterogeneous high-dimensional modalities—including images, LiDAR point clouds, and IMU signals. Evaluated on real-world road tests and the CrisisMMD dataset, our method achieves 96.42% detection accuracy, an inference latency of only 0.0351 seconds per sample, and reduces communication overhead to 1/1000 of comparable approaches. These advances significantly enhance model practicality, real-time performance, and privacy guarantees in safety-critical vehicular environments.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Federated Web and WoT systems, including distributed, federated and edge-based data processing
• User Modeling, Personalization and Recommendation: Federated recommendation systems and personalization
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

Internet of Things (IoTs) have been widely applied in Collaborative Intelligent Transportation Systems (C-ITS) for the prevention of road accidents. As one of the primary causes of road accidents in C-ITS, the efficient detection and early alarm of road hazards are of paramount importance. Given the importance, extensive research has explored this topic and obtained favorable results. However, most existing solutions only explore single-modality data, struggle with high computation and communication overhead, or suffer from the curse of high dimensionality in their privacy-preserving methodologies. To overcome these obstacles, in this paper, we introduce RoadFed, an innovative and private multimodal Federated learning-based system tailored for intelligent Road hazard detection and alarm. This framework encompasses an innovative Multimodal Road Hazard Detector, a communication-efficient federated learning approach, and a customized low-error-rate local differential privacy method crafted for high dimensional multimodal data. Experimental results reveal that the proposed RoadFed surpasses most existing systems in the self-gathered real-world and CrisisMMD public datasets. In particular, RoadFed achieves an accuracy of 96.42% with a mere 0.0351 seconds of latency and its communication cost is up to 1,000 times lower than existing systems in this field. It facilitates collaborative training with non-iid high dimensional multimodal real-world data across various data modalities on multiple edges while ensuring privacy preservation for road users.

Problem

• Multimodal Federated Learning
• Road Hazard Detection
• Privacy-Preserving Methodology

Innovation

• Multimodal Federated Learning System
• Communication-efficient Learning Approach
• Low-error-rate Local Differential Privacy