Metrics vs Surveys: Can Quantitative Measures Replace Human Surveys in Social Robot Navigation? A Correlation Analysis

2025-10-03

AI Summary

Human-perception evaluation of social robot navigation—encompassing comfort, safety, and intelligibility—has long relied on costly, non-reproducible user studies, while existing quantitative metrics lack standardization and exhibit unclear correlations with subjective experience. Method: This paper systematically analyzes statistical correlations between prevalent objective metrics (e.g., minimum distance, velocity, avoidance frequency) and large-scale human survey data across diverse social navigation scenarios. Contribution/Results: We reveal fundamental limitations: current metrics show only weak correlations with comfort and intelligibility, and negligible predictive power for perceived safety. Based on these findings, we propose principled guidelines for designing human-perception-aware metrics and empirically validate their necessity. Results demonstrate that optimizing conventional metrics alone cannot substitute for human evaluation. Instead, a new measurement framework—integrating behavioral semantics and cognitive models—is essential to enable reproducible, comparable, and psychologically grounded evaluation of social navigation.

Technology Category

• Natural Language Processing: Safety and Robustness
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Intelligent Robots: Human-Robot Interaction

Application Category

• Search and Retrieval-Augmented AI: Web evaluation methodologies and metrics
• User Modeling, Personalization and Recommendation: Metrics for user behavior and evaluating success
• Security and Privacy: Large-scale security measurements

Abstract

Social, also called human-aware, navigation is a key challenge for the integration of mobile robots into human environments. The evaluation of such systems is complex, as factors such as comfort, safety, and legibility must be considered. Human-centered assessments, typically conducted through surveys, provide reliable insights but are costly, resource-intensive, and difficult to reproduce or compare across systems. Alternatively, numerical social navigation metrics are easy to compute and facilitate comparisons, yet the community lacks consensus on a standard set of metrics. This work explores the relationship between numerical metrics and human-centered evaluations to identify potential correlations. If specific quantitative measures align with human perceptions, they could serve as standardized evaluation tools, reducing the dependency on surveys. Our results indicate that while current metrics capture some aspects of robot navigation behavior, important subjective factors remain insufficiently represented and new metrics are necessary.

Problem

• Investigating if quantitative metrics can replace human surveys in robot navigation evaluation
• Analyzing correlation between numerical measures and human perceptions of social navigation
• Identifying limitations of current metrics in capturing subjective factors like comfort

Innovation

• Correlating numerical metrics with human surveys
• Identifying quantitative measures for subjective factors
• Proposing new metrics for human-aware navigation