Calibration and Evaluation of Car-Following Models for Autonomous Shuttles Using a Novel Multi-Criteria Framework

2026-02-12

AI Summary

This study addresses the lack of a dedicated calibration and unified evaluation framework for car-following models tailored to autonomous shuttle vehicles, which has hindered a deeper understanding of their traffic behavior. Leveraging real-world shuttle trajectory data, the authors systematically calibrate eight machine learning algorithms—including XGBoost, LSTM, and CNN—as well as two physics-based models (IDM and ACC), and propose a multi-criteria evaluation framework integrating prediction accuracy, trajectory stability, and statistical similarity. Results demonstrate that the calibrated XGBoost model achieves the best overall performance, while LSTM and CNN exhibit strong long-term positional stability but weaker short-term dynamic responsiveness. Conventional models show comparatively lower accuracy and stability. The proposed generalizable framework significantly enhances inter-model comparability and reproducibility.

Technology Category

• Machine Learning: Calibration & Uncertainty Quantification
• Planning, Routing, and Scheduling: Model-Based Reasoning
• Natural Language Processing: Interpretability, Analysis, and Evaluation of NLP Models

Application Category

• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Autonomous shuttles (AS) are fully autonomous transit vehicles with operating characteristics distinct from conventional autonomous vehicles (AV). Developing dedicated car-following models for AS is critical to understanding their traffic impacts; however, few studies have calibrated such models with field data. More advanced machine learning (ML) techniques have not yet been applied to AS trajectories, leaving the potential of ML for capturing AS dynamics unexplored and constraining the development of dedicated AS models. Furthermore, there is a lack of a unified framework for systematically evaluating and comparing the performance of car-following models to replicate real trajectories. Existing car-following studies often rely on disparate metrics, which limit reproducibility and performance comparability. This study addresses these gaps through two main contributions: (1) the calibration of a diverse set of car-following models using real-world AS trajectory data, including eight machine learning algorithms and two physics-based models; and (2) the introduction of a multi-criteria evaluation framework that integrates measures of prediction accuracy, trajectory stability, and statistical similarity, which provides a generalizable methodology for a systematic assessment of car-following models. Results indicated that the proposed calibrated XGBoost model achieved the best overall performance. Sequential model type, such as LSTM and CNN, captured long-term positional stability but were less responsive to short-term dynamics. LSTM and CNN captured long-term positional stability but were less responsive to short-term dynamics. Traditional models (IDM, ACC) and kernel methods showed lower accuracy and stability than most ML models tested.

Problem

• autonomous shuttles
• car-following models
• model calibration
• trajectory evaluation
• multi-criteria framework

Innovation

• car-following models
• autonomous shuttles
• machine learning
• multi-criteria evaluation
• trajectory calibration