TriDrive: Joint Driver, Vehicle, and Road Modeling for Forecasting and Driver Monitoring

2026-09-26

AI Summary

This study addresses the limitations of isolated modeling in driver-vehicle-road interaction prediction by proposing the first unified tri-modal joint prediction framework. Methodologically, it achieves multi-modal collaborative prediction through an automated conditional transition model and introduces directed residual connections to integrate driver and road context for enhanced vehicle trajectory forecasting. The architecture employs modality-specific encoders—anchored kinematics, causal CAN bus signals, and frozen V-JEPA 2 road latents—augmented with knowledge distillation techniques. Experimental evaluations demonstrate that the proposed approach achieves state-of-the-art performance on the AIDE benchmark, with a real-world deployment latency of merely 177 ms. These results indicate substantial improvements in both the timeliness and accuracy of advanced driver assistance system warnings, validating the framework’s practical efficacy for real-time vehicular applications.

Abstract

Predicting how drivers, vehicles, and road scenes interact and evolve together is central to driver monitoring. Prior work models in-cabin activity or traffic-conditioned driver motion in isolation, motivating joint driver, vehicle, and road modeling with real-time on-vehicle evaluation. We introduce TriDrive, to our knowledge the first unified framework that jointly forecasts driver kinematics, vehicle dynamics, and road demands through an automation-conditioned transition model. Modality-specific encoders (an anchored kinematic representation of the driver, causal CAN-bus dynamics, and frozen V-JEPA 2 road latents with structured road margins) are connected by directed residual connections through which driver and road context refine vehicle forecasts. We evaluate TriDrive on three downstream tasks. On the public AIDE benchmark, its kinematic encoder recipe sets a new full-set state of the art (SOTA) among published baselines (48.05 versus 71.47 All-MPJPE). On 197.2 hours of naturalistic BATON subset, directed connections and road margins raise assistance-engaged PR-AUC by 0.084 for steering onset and 0.286 for time-to-collision drops. For real-time use, we distill the road encoders and run TriDrive on a comma four with an external 8 GB GPU, where a lightweight current-state warning probe updates at 5 Hz with 177 ms p95 latency while the joint model forecasts concurrently. The probe is above an openpilot-based baseline on human-labeled manual-driving warnings (AUROC 0.725 versus 0.563), and in a paired on-road study 14 drivers rate its warnings as more appropriate (+1.79) and timely (+2.67) than those of openpilot's driver-monitoring system.

Problem

• driver monitoring
• joint modeling
• trajectory forecasting
• real-time evaluation

Innovation

• Joint Modeling
• Driver Monitoring
• Trajectory Forecasting
• Knowledge Distillation
• Real-time Deployment