Ranking and Selection with Simultaneous Input Data Collection

2025-03-14

AI Summary

This paper addresses the sequential selection problem under streaming heterogeneous inputs, where multi-source data collection and simulation execution must be coordinated under constrained budgets. Method: We propose the first synchronous budget allocation framework that constructs a performance estimator based on temporally aggregated heterogeneous simulation outputs and jointly optimizes data acquisition and simulation resource allocation. Theoretically, we establish asymptotic consistency and asymptotic normality of the estimator. Methodologically, we design a multi-stage stochastic optimization algorithm ensuring both statistical reliability and computational tractability. Results: Numerical experiments demonstrate that our approach significantly outperforms existing benchmarks in selection accuracy and resource utilization efficiency, providing a provably sound, computationally feasible, and practically deployable paradigm for real-time sequential decision-making under streaming heterogeneous environments.

Technology Category

• Search and Optimization: Sampling/Simulation-based Search
• Reasoning under Uncertainty: Stochastic Optimization
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Economics, Online Markets and Human Computation: Incentives in network design for Web infrastructures and ecosystems
• Systems and Infrastructure for Web, Mobile and WoT: Data management and stream processing for Web, mobile and wireless applications
• Web Mining and Content Analysis: Web data generation and simulation

Abstract

In this paper, we propose a general and novel formulation of ranking and selection with the existence of streaming input data. The collection of multiple streams of such data may consume different types of resources, and hence can be conducted simultaneously. To utilize the streaming input data, we aggregate simulation outputs generated under heterogeneous input distributions over time to form a performance estimator. By characterizing the asymptotic behavior of the performance estimators, we formulate two optimization problems to optimally allocate budgets for collecting input data and running simulations. We then develop a multi-stage simultaneous budget allocation procedure and provide its statistical guarantees such as consistency and asymptotic normality. We conduct several numerical studies to demonstrate the competitive performance of the proposed procedure.

Problem

• Optimize budget allocation for simultaneous data collection and simulations.
• Develop performance estimators from heterogeneous input distributions over time.
• Ensure statistical guarantees like consistency and asymptotic normality.

Innovation

• Simultaneous collection of multiple data streams
• Aggregation of simulation outputs over time
• Multi-stage budget allocation with statistical guarantees