PreGSU-A Generalized Traffic Scene Understanding Model for Autonomous Driving based on Pre-trained Graph Attention Network

2024-04-16

arXiv.org

AI Summary

Current autonomous driving scene understanding methods are predominantly task-specific, exhibiting limited generalization capability and struggling to adapt to complex traffic environments and diverse downstream tasks. To address this, we propose the first generalized model for universal traffic scene understanding, which uniformly models dynamic agent-agent and agent-road interactions using a graph attention network. We introduce two novel self-supervised pretraining paradigms: Virtual Interaction Force (VIF) and Masked Road Modeling (MRM), enabling unified representation learning and generalizable relational reasoning over traffic dynamics. Our method achieves state-of-the-art performance on urban trajectory prediction and highway intention recognition, consistently outperforming dedicated single-task baselines. Moreover, it demonstrates strong cross-dataset generalization. Ablation studies confirm the effectiveness of the proposed pretraining mechanisms.

Technology Category

• Planning, Routing, and Scheduling: Model-Based Reasoning
• Computer Vision: Large Vision Models
• Machine Learning: Unsupervised & Self-Supervised Learning

Application Category

• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Scene understanding, defined as learning, extraction, and representation of interactions among traffic elements, is one of the critical challenges toward high-level autonomous driving (AD). Current scene understanding methods mainly focus on one concrete single task, such as trajectory prediction and risk level evaluation. Although they perform well on specific metrics, the generalization ability is insufficient to adapt to the real traffic complexity and downstream demand diversity. In this study, we propose PreGSU, a generalized pre-trained scene understanding model based on graph attention network to learn the universal interaction and reasoning of traffic scenes to support various downstream tasks. After the feature engineering and sub-graph module, all elements are embedded as nodes to form a dynamic weighted graph. Then, four graph attention layers are applied to learn the relationships among agents and lanes. In the pre-train phase, the understanding model is trained on two self-supervised tasks: Virtual Interaction Force (VIF) modeling and Masked Road Modeling (MRM). Based on the artificial potential field theory, VIF modeling enables PreGSU to capture the agent-to-agent interactions while MRM extracts agent-to-road connections. In the fine-tuning process, the pre-trained parameters are loaded to derive detailed understanding outputs. We conduct validation experiments on three datasets and two downstream tasks, i.e., trajectory prediction in urban scenario and intention recognition in highway scenario, to verify the model's generalization and understanding capabilities. Results show that compared with single-task-driven baselines, PreGSU achieves competitive performance on all datasets and downstream tasks, indicating its potential to be generalized to various scenes and targets. Ablation study shows the effectiveness of pre-train task design.

Problem

• Generalized scene understanding for autonomous driving complexity
• Learning universal traffic interactions via graph attention network
• Adapting pre-trained model to diverse downstream tasks

Innovation

• Pre-trained graph attention network for traffic scenes
• Self-supervised Virtual Interaction Force modeling
• Masked Road Modeling for agent-road connections