Realistic pedestrian-driver interaction modelling using multi-agent RL with human perceptual-motor constraints

2025-10-31

AI Summary

Existing pedestrian-driver interaction models often neglect human perceptual-motor constraints, leading to unrealistic behaviors and compromised safety. This paper addresses unsignalized intersection scenarios and proposes the first multi-agent reinforcement learning framework that jointly models visual constraints (e.g., field-of-view limitations, perceptual noise) and motor constraints (e.g., action smoothness, response latency). Crucially, it introduces a novel unified representation of both constraint types as a population-level distribution—a first in the literature. The method significantly enhances behavioral fidelity and interactive safety: on real-world datasets, the joint-constraint model achieves state-of-the-art performance, generating smoother, more cautious trajectories. Notably, it substantially outperforms behavior cloning baselines under low-data regimes, demonstrating robust generalization with limited demonstration samples.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Multiagent Systems: Adversarial Agents
• Planning, Routing, and Scheduling: Model-Based Reasoning

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: User modeling and simulation for interactive and conversational systems

Abstract

Modelling pedestrian-driver interactions is critical for understanding human road user behaviour and developing safe autonomous vehicle systems. Existing approaches often rely on rule-based logic, game-theoretic models, or 'black-box' machine learning methods. However, these models typically lack flexibility or overlook the underlying mechanisms, such as sensory and motor constraints, which shape how pedestrians and drivers perceive and act in interactive scenarios. In this study, we propose a multi-agent reinforcement learning (RL) framework that integrates both visual and motor constraints of pedestrian and driver agents. Using a real-world dataset from an unsignalised pedestrian crossing, we evaluate four model variants, one without constraints, two with either motor or visual constraints, and one with both, across behavioural metrics of interaction realism. Results show that the combined model with both visual and motor constraints performs best. Motor constraints lead to smoother movements that resemble human speed adjustments during crossing interactions. The addition of visual constraints introduces perceptual uncertainty and field-of-view limitations, leading the agents to exhibit more cautious and variable behaviour, such as less abrupt deceleration. In this data-limited setting, our model outperforms a supervised behavioural cloning model, demonstrating that our approach can be effective without large training datasets. Finally, our framework accounts for individual differences by modelling parameters controlling the human constraints as population-level distributions, a perspective that has not been explored in previous work on pedestrian-vehicle interaction modelling. Overall, our work demonstrates that multi-agent RL with human constraints is a promising modelling approach for simulating realistic road user interactions.

Problem

• Simulates realistic pedestrian-driver interactions using multi-agent reinforcement learning
• Incorporates human visual and motor constraints to improve behavioral realism
• Models individual differences through population-level constraint parameter distributions

Innovation

• Multi-agent RL with human perceptual-motor constraints
• Combined visual and motor constraints for realistic behavior
• Population-level distributions for individual differences modeling