Rethinking Sales Lead Scoring with LLM-based Hierarchical Preference Ranking

2026-06-02

AI Summary

This work addresses key challenges in scoring high-value sales leads—namely, prolonged decision cycles, sparse labels, difficulty in modeling unstructured interaction semantics, and the lack of quantifiable lead prioritization—by proposing a discriminative scoring framework grounded in large language models (LLMs). The framework integrates structured CRM features with unstructured customer text and introduces a novel hierarchical preference ranking optimization (HPRO) method, which transforms sparse binary labels into preference pairs encoding funnel-stage information. Joint learning leverages both pointwise and pairwise supervision signals, enabled by funnel-aware preference construction and a margin-aware Bradley–Terry model, marking the first effective discriminative application of LLMs to sales lead ranking. Evaluated on real-world data from a leading new-energy vehicle brand, the model achieves an AUC of 0.8161, improves top-lead ranking precision by 39.7%, and drives a 9.5% increase in revenue over a 132-day online A/B test, demonstrating substantial commercial impact.

Abstract

Sales lead conversion in high-stakes domains (e.g., automotive, real estate) differs fundamentally from e-commerce recommendation due to prolonged decision cycles and multi-stage funnels. Traditional lead scoring methods rule-based scorecards, machine learning, or pointwise CTR models face severe challenges: sparse supervision, a semantic gap in unstructured CRM logs, and inability to capture relative lead priority. While Large Language Models(LLMs) offer superior semantic understanding of customer interactions, general-purpose LLMs are ill-suited for lead ranking: they generate text rather than comparable scores, and lack alignment with the hierarchical priorities of sales funnels. We introduce an LLM-based discriminative framework for sales lead scoring, which supports joint modeling of structured CRM features and unstructured customer interactions. On top of this framework, we propose HPRO (Hierarchical Preference Ranking Optimization), which augments sales lead scoring with a hierarchical preference ranking objective. HPRO employs a margin-aware Bradley-Terry formulation to transform sparse binary labels into dense, funnel-aware preference pairs, enabling lead scoring to leverage both pointwise and pairwise supervision. Experiments on large-scale data from a leading NEV brand demonstrate state-of-the-art classification (AUC 0.8161) and ranking performance (+39.7% precision among top-ranked leads). A 132-day online A/B test validates 9.5% sales volume uplift, confirming real-world commercial impact.

Problem

• lead scoring
• sales funnel
• hierarchical preference
• sparse supervision
• semantic gap

Innovation

• LLM-based lead scoring
• Hierarchical Preference Ranking
• HPRO
• Bradley-Terry model
• Sales funnel alignment