SYNAPSE: SYmbolic Neural-Aided Preference Synthesis Engine

2024-03-25

AI Summary

This work addresses the challenge of aligning robotic behavior with users’ subjective preferences—e.g., “ideal parallel parking position”—by proposing a novel, few-shot visual demonstration–based method for modeling individualized preferences. The core method introduces the first neural-symbolic preference program representation: scene semantics are extracted via visual parsing; natural-language preference descriptions are interpreted using large language models; and interpretable programs are synthesized and reasoned over within a domain-specific language (DSL). The framework enables preference debugging, cross-user generalization, and out-of-distribution robustness. Evaluated on diverse preference-driven tasks and real-user studies, it significantly outperforms baseline approaches. Ablation studies confirm the efficacy of each component—visual parsing, LLM-guided DSL synthesis, and neural-symbolic reasoning—demonstrating their complementary roles in achieving accurate, interpretable, and adaptable preference modeling.

Technology Category

• Knowledge Representation and Reasoning: Preferences
• Humans and AI: Learning Human Values and Preferences
• Machine Learning: Learning Preferences or Rankings

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• User Modeling, Personalization and Recommendation: Explainable and interpretable methods for personalization
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

This paper addresses the problem of preference learning, which aims to align robot behaviors through learning user specific preferences (e.g."good pull-over location") from visual demonstrations. Despite its similarity to learning factual concepts (e.g."red door"), preference learning is a fundamentally harder problem due to its subjective nature and the paucity of person-specific training data. We address this problem using a novel framework called SYNAPSE, which is a neuro-symbolic approach designed to efficiently learn preferential concepts from limited data. SYNAPSE represents preferences as neuro-symbolic programs, facilitating inspection of individual parts for alignment, in a domain-specific language (DSL) that operates over images and leverages a novel combination of visual parsing, large language models, and program synthesis to learn programs representing individual preferences. We perform extensive evaluations on various preferential concepts as well as user case studies demonstrating its ability to align well with dissimilar user preferences. Our method significantly outperforms baselines, especially when it comes to out of distribution generalization. We show the importance of the design choices in the framework through multiple ablation studies. Code, additional results, and supplementary material can be found on the website: https://amrl.cs.utexas.edu/synapse

Problem

• User-specific Preferences
• Visual Demonstrations
• Sparse Training Data

Innovation

• Neuro-symbolic framework
• Subjective preferences learning
• Out-of-distribution generalization