Selected work

Representative Papers

The Noise Is the Signal: Correlated Sampling Error Is Rank-Informative for Proxy Metric Selection

Oct 06, 2026

This study addresses the challenge of surrogate metric selection in A/B testing, where high noise and limited sample sizes compromise the reliability of north star metrics. Departing from conventional approaches that treat shared sampling error between surrogate and north star metrics as contamination requiring correction, this work proposes a paradigm-shifting "noise-as-signal" framework. We demonstrate that such shared errors encode critical ranking information. Through disjoint subset estimation, Spearman rank correlation analysis, and validation across 262 real-world experiments, we find that shared sampling error aligns strongly with true metric rankings (correlation coefficient of 0.65), and removing it significantly degrades surrogate ranking accuracy. This research establishes the positive value of error correlation for model selection, offering online platforms a cost-effective evaluation strategy for surrogate metric assessment.

0 citationsRead paper

On Evaluating and Improving Conversational Agents in Production

Sep 25, 2026

This study addresses the challenges of non-replayable logs, systemic stochastic fluctuations, and metrics that fail to localize behavioral changes in the offline evaluation of large-scale multi-agent shopping assistants. To overcome these issues, we propose a systematic evaluation and improvement framework. Methodologically, it employs grounded user simulation to generate dialogues as an alternative to log replay, utilizes repeated baseline runs with paired percentile bootstrapping to mitigate noise, and introduces fixed-scenario assertion-based mechanisms for hypothesis-driven fine-tuning verification. In practice, this framework successfully localized a product carousel malfunction, effectively distinguished genuine improvements from runtime variance, and uncovered missing evidence alongside configuration failures, thereby substantially enhancing debugging efficiency in production environments.

0 citationsRead paper

Measuring Opportunity Cost with Stock Lifetime Value

Jul 02, 2026

This study addresses the limitation of conventional e-commerce A/B tests, which often overlook the long-term impact of interventions on profitability across an inventory item’s full lifecycle due to short experimental windows. To overcome this, the authors propose Stock Lifetime Value (SLV), a novel metric that aggregates the expected profit of current inventory over its entire sales horizon within short-term experiments, thereby enabling more accurate assessment of long-term profitability. SLV uniquely integrates inventory constraints and seasonal lifecycle dynamics into the A/B testing framework, combining causal inference with financial mapping to support both item-level and user-level experimentation while aligning with annual financial reporting. Empirical validation at Zalando demonstrates that SLV effectively predicts actual profits over an 18-month horizon, enhances pricing algorithm performance, and delivers interpretable estimates of annual financial impact.

0 citationsRead paper

VCG: A Multimodal Retrieval Framework for E-Commerce Video Feeds under Extreme Cold-Start Conditions

Jun 17, 2026

This work addresses the challenges of user interaction scarcity and positional/duration biases in e-commerce short videos under extreme cold-start scenarios by proposing the first zero-shot multimodal retrieval framework tailored for e-commerce video streams. Built upon a domain-adapted CLIP vision-language model, the framework aligns user intent and video content into a shared semantic space, enabling bidirectional product-to-video and semantic retrieval. It further provides a systematic comparison between discriminative (CLIP-based) and generative (LLM-based) embedding strategies to optimize candidate generation. Online A/B testing demonstrates that the proposed approach significantly improves deep video completion rates by 50%, validating its effectiveness and scalability in large-scale real-world e-commerce environments.

0 citationsRead paper

High-Frequency Pricing at Scale for E-Commerce

Jun 11, 2026

This study addresses the challenges of extreme demand volatility, delayed pricing responses, and misalignment between short-term revenue and long-term profitability during major fashion e-commerce promotions. To tackle these issues, the authors propose a high-frequency “predict–optimize” automated pricing system that breaks away from traditional weekly decision cycles by operating at a minute-level granularity. The system achieves the first industrial-scale deployment of daily multi-objective dynamic pricing in large-scale e-commerce settings, combining gradient-boosted tree models for daily demand forecasting with a multi-objective optimization framework to generate real-time pricing strategies that jointly maximize long-term profit and net merchandise value. Evaluated across 23 A/B tests in 12 Zalando markets from 2023 to 2024, the system delivered approximately 6% higher profit while maintaining sales volume and has since been fully deployed for promotional pricing.

0 citationsRead paper

Recent publications

Latest Papers

The Noise Is the Signal: Correlated Sampling Error Is Rank-Informative for Proxy Metric Selection

Oct 06, 2026

This study addresses the challenge of surrogate metric selection in A/B testing, where high noise and limited sample sizes compromise the reliability of north star metrics. Departing from conventional approaches that treat shared sampling error between surrogate and north star metrics as contamination requiring correction, this work proposes a paradigm-shifting "noise-as-signal" framework. We demonstrate that such shared errors encode critical ranking information. Through disjoint subset estimation, Spearman rank correlation analysis, and validation across 262 real-world experiments, we find that shared sampling error aligns strongly with true metric rankings (correlation coefficient of 0.65), and removing it significantly degrades surrogate ranking accuracy. This research establishes the positive value of error correlation for model selection, offering online platforms a cost-effective evaluation strategy for surrogate metric assessment.

0 citationsRead paper

On Evaluating and Improving Conversational Agents in Production

Sep 25, 2026

This study addresses the challenges of non-replayable logs, systemic stochastic fluctuations, and metrics that fail to localize behavioral changes in the offline evaluation of large-scale multi-agent shopping assistants. To overcome these issues, we propose a systematic evaluation and improvement framework. Methodologically, it employs grounded user simulation to generate dialogues as an alternative to log replay, utilizes repeated baseline runs with paired percentile bootstrapping to mitigate noise, and introduces fixed-scenario assertion-based mechanisms for hypothesis-driven fine-tuning verification. In practice, this framework successfully localized a product carousel malfunction, effectively distinguished genuine improvements from runtime variance, and uncovered missing evidence alongside configuration failures, thereby substantially enhancing debugging efficiency in production environments.

0 citationsRead paper

Measuring Opportunity Cost with Stock Lifetime Value

Jul 02, 2026

This study addresses the limitation of conventional e-commerce A/B tests, which often overlook the long-term impact of interventions on profitability across an inventory item’s full lifecycle due to short experimental windows. To overcome this, the authors propose Stock Lifetime Value (SLV), a novel metric that aggregates the expected profit of current inventory over its entire sales horizon within short-term experiments, thereby enabling more accurate assessment of long-term profitability. SLV uniquely integrates inventory constraints and seasonal lifecycle dynamics into the A/B testing framework, combining causal inference with financial mapping to support both item-level and user-level experimentation while aligning with annual financial reporting. Empirical validation at Zalando demonstrates that SLV effectively predicts actual profits over an 18-month horizon, enhances pricing algorithm performance, and delivers interpretable estimates of annual financial impact.

0 citationsRead paper

VCG: A Multimodal Retrieval Framework for E-Commerce Video Feeds under Extreme Cold-Start Conditions

Jun 17, 2026

This work addresses the challenges of user interaction scarcity and positional/duration biases in e-commerce short videos under extreme cold-start scenarios by proposing the first zero-shot multimodal retrieval framework tailored for e-commerce video streams. Built upon a domain-adapted CLIP vision-language model, the framework aligns user intent and video content into a shared semantic space, enabling bidirectional product-to-video and semantic retrieval. It further provides a systematic comparison between discriminative (CLIP-based) and generative (LLM-based) embedding strategies to optimize candidate generation. Online A/B testing demonstrates that the proposed approach significantly improves deep video completion rates by 50%, validating its effectiveness and scalability in large-scale real-world e-commerce environments.

0 citationsRead paper

High-Frequency Pricing at Scale for E-Commerce

Jun 11, 2026

This study addresses the challenges of extreme demand volatility, delayed pricing responses, and misalignment between short-term revenue and long-term profitability during major fashion e-commerce promotions. To tackle these issues, the authors propose a high-frequency “predict–optimize” automated pricing system that breaks away from traditional weekly decision cycles by operating at a minute-level granularity. The system achieves the first industrial-scale deployment of daily multi-objective dynamic pricing in large-scale e-commerce settings, combining gradient-boosted tree models for daily demand forecasting with a multi-objective optimization framework to generate real-time pricing strategies that jointly maximize long-term profit and net merchandise value. Evaluated across 23 A/B tests in 12 Zalando markets from 2023 to 2024, the system delivered approximately 6% higher profit while maintaining sales volume and has since been fully deployed for promotional pricing.

0 citationsRead paper