DIRECTOR: Dynamic Index-based Recommendation with Transport-Optimized Retrieval

2026-07-28

AI Summary

This work addresses key limitations in existing re-ranking methods: autoregressive models suffer from high latency and constrained search spaces, while non-autoregressive approaches exhibit weak cross-position coordination and a tendency to generate redundant recommendations. To overcome these challenges, the paper proposes a parallel re-ranking framework guided by optimal transport, which leverages dynamic retrieval indexing to achieve global structural coherence and efficient, duplication-free generation. The core innovations include entropy-regularized optimal transport for conflict-aware training, a prefix-anchored credit assignment mechanism that decomposes list-level rewards into position-level signals, and the integration of continuous latent space mapping with hard-matching inference. Evaluated on large-scale industrial recommendation scenarios, the proposed method significantly outperforms current re-ranking baselines, with both offline and online experiments confirming its effectiveness.

Abstract

Reranking is a combinatorial decision problem that aims to select and order a high-utility slate from a request-specific candidate set. A major line of generative rerankers adopts autoregressive (AR) models, which construct the slate one position at a time to capture inter-position dependencies. However, under practical greedy or bounded-width decoding, prefix-based search may prematurely prune globally promising permutations and incurs inherently sequential latency, restricting the effective search space under a fixed serving budget. Non-autoregressive (NAR) alternatives alleviate this efficiency bottleneck through position-parallel prediction, but naive position-wise factorization treats different positions too independently, leading to insufficient cross-position coordination and potentially duplicate or conflicting item selections. To retain parallel efficiency while introducing global structural coordination, we propose Dynamic Index-based RECommendation with Transport-Optimized Retrieval (DIRECTOR), a transport-guided parallel reranking framework. DIRECTOR maps candidate items into a continuous latent space and generates request-conditioned dynamic retrieval indices for all target positions in parallel. During training, it uses entropy-regularized OT to provide conflict-aware supervision; at inference, it directly performs global hard matching on similarity matrix, producing duplicate-free slates without iterative transport. To further align the generator with an opaque list-wise evaluator that returns only a scalar utility, we introduce a prefix-anchored credit assignment mechanism that converts the global reward into position-specific training signals. Extensive offline and online experiments demonstrate that DIRECTOR consistently outperforms strong reranking baselines, achieving significant improvement in large-scale industrial recommendation scenarios.

Problem

• reranking
• autoregressive models
• non-autoregressive models
• cross-position coordination
• duplicate-free slate

Innovation

• non-autoregressive reranking
• optimal transport
• dynamic indexing
• parallel recommendation
• credit assignment