From Profiling to Synthesis: Benchmarking Implicit Behavioral Alignment in Personalized LLM Agents

2026-08-03

AI Summary

This work addresses the limitations of existing evaluation methods for personalized agents, which rely on static preferences or fixed interaction logs and thus fail to capture the dynamic evolution of user preferences or assess the agent’s ability to translate preferences into task execution. To bridge this gap, we introduce IBA-Bench—the first dynamic benchmark for implicit behavior alignment—that evaluates agents’ capacity to satisfy implicit user constraints during task execution based on long-term interaction histories characterized by noise, implicit cues, and temporal inconsistencies. We further propose the IBA-Agent framework, which resolves conflicting preferences through broad-context retrieval and trajectory-level alignment mechanisms. Experiments across nine domains demonstrate that prevailing LLM-based agents underperform on implicit alignment tasks, whereas IBA-Agent significantly enhances personalized task execution in complex scenarios.

Abstract

Large Language Models have enabled increasingly capable autonomous agents, yet personalization remains critical for making such agents practically useful. Recent benchmarks have begun evaluating personalization in agents, but they largely rely on static preference snapshots, fixed interaction logs, or question answering over predefined user profiles. Such designs fail to capture the complexity of evolving user preferences and neglect preference-conditioned task execution-a discrepancy we term as the knowledge-to-action gap. To address this challenge, we introduce IBA-Bench, a benchmark for implicit behavioral alignment constructed from longitudinal interaction histories that contain noise, implicit cues, and temporal inconsistencies. Unlike prior work, IBA-Bench evaluates whether an agent can execute tasks while satisfying implicit user constraints inferred from historical interactions. We further propose IBA-Agent, an agent framework that reconciles conflicting priorities through broad retrieval and trajectory-level alignment. Experiment results on IBA-Bench show that effective personalization remains a significant challenge for state-of-the-art LLM agents, and the proposed IBA-Agent substantially improves behavioral alignment in complex scenarios across nine application domains.

Problem

• personalized LLM agents
• implicit behavioral alignment
• knowledge-to-action gap
• evolving user preferences
• preference-conditioned task execution

Innovation

• implicit behavioral alignment
• personalized LLM agents
• longitudinal interaction histories
• trajectory-level alignment
• IBA-Bench