RobotValues: Evaluating Household Robots When Human Values Conflict

2026-06-02

AI Summary

This work addresses the narrow focus of existing home robot evaluation frameworks, which prioritize task completion while overlooking trade-offs among diverse human values such as autonomy and privacy. To bridge this gap, we introduce RobotValues—the first benchmark for assessing value conflicts in domestic robots—comprising 10,000 realistic, image-driven household scenarios, each offering multiple action choices aligned with different values. The benchmark is constructed through stakeholder-informed value elicitation, LLM-assisted scenario generation, and AI-synthesized imagery, and leverages vision-language models for decision evaluation. Our experiments reveal that current models exhibit strong default value preferences—typically favoring safety and compliance—and fail to adapt to non-default values like privacy in approximately 80% of cases, highlighting a critical limitation in their ability to achieve nuanced value alignment.

Abstract

While household robots are often evaluated based on task completion, everyday domestic environments involve value-conflicting situations in which robots are expected to choose actions that prioritize other values than task success, such as human autonomy, efficiency, or social appropriateness. Yet, there are no benchmarks for evaluating robots' value preferences in such scenarios. We introduce RobotValues, a benchmark to evaluate household robot planners in 10K value-conflict scenarios. Each instance consists of a realistic household image with multiple plausible robot actions that prioritize different human values. We construct RobotValues through LLM-assisted scenario generation, stakeholder-grounded value extraction, image generation and automatic quality control. Using RobotValues we evaluate VLMs used in robotics and find that models exhibit default value preferences, including safety and accommodation, while underselecting privacy-prioritizing actions. When the models are instructed to prioritize specific values that conflict with their own preferences, they often fail to override their default actions, choosing incorrect actions for 80% of the time. These findings suggest that household robot evaluation should measure not only task completion or safety compliance, but also whether robots can choose among plausible actions when human values conflict.

Problem

• value conflict
• household robots
• robot evaluation
• human values
• ethical decision-making

Innovation

• value conflict
• robot benchmark
• human values
• vision-language models
• ethical decision-making