PrefBench: Evaluating Zero-Shot LLM Agents in Hidden-Preference Personalized Pricing Negotiations

2026-05-19

AI Summary

This work addresses the challenge of profit optimization in personalized pricing negotiations under hidden buyer preferences, where existing zero-shot LLM agents achieve high deal-closure rates but fail to maximize revenue. To disentangle negotiation success from profit sensitivity, we introduce PrefBench—the first structured negotiation simulation benchmark tailored for hidden preferences—featuring a strict JSON-based action protocol and fixed information boundaries. PrefBench integrates latent-variable-driven buyer models, structured state summaries, and heuristic baselines. Experiments reveal that while mainstream LLMs attain deal rates exceeding 0.99, their average profits barely surpass those of random strategies and fall significantly short of a simple concession-based heuristic, demonstrating that high compliance does not imply high profitability.

Abstract

Personalized pricing negotiations are a challenging testbed for LLM agents because successful interaction does not guarantee profitable decision making. A seller may produce valid actions and close many deals while still pricing poorly when buyer willingness to pay and bargaining traits remain hidden. This paper presents PrefBench, a simulator-based benchmark for hidden-preference personalized pricing negotiations. Each episode pairs a simulated buyer with a fixed vehicle-customization bundle; the seller observes public persona descriptors, bundle information, and negotiation history, while latent buyer variables govern valuation, patience, counter-offer behavior, and walkaway decisions. PrefBench evaluates this setting through an LLM-facing state-summary protocol that constrains agents to return strict JSON actions under a fixed hidden-information boundary. We evaluate zero-shot LLM sellers against heuristic references over 7,500 episodes. The tested LLMs follow the protocol reliably and achieve deal rates above 0.99, but their seller-profit outcomes remain weak: the best LLM average profit is only slightly above the random baseline and far below a simple concession heuristic under the same episode stream. These results show that structured action compliance and agreement-seeking behavior can coexist with weak profit-sensitive bargaining. PrefBench provides a controlled benchmark for evaluating pricing-agent behavior under hidden buyer preferences.

Problem

• personalized pricing
• hidden preferences
• LLM agents
• negotiation
• zero-shot

Innovation

• personalized pricing
• hidden preferences
• LLM agents
• negotiation benchmark
• zero-shot evaluation