extit{From Freshness to Effectiveness}: Goal-Oriented Sampling for Remote Decision Making

2025-04-28

AI Summary

In remote decision-making scenarios—such as V2X, smart healthcare, and industrial IoT—the Age of Information (AoI) fails to fully capture data’s decision-relevance; outdated data may sometimes exhibit higher discriminative power. Method: We propose the Age-aware Remote Markov Decision Process (AR-MDP) framework, jointly optimizing data sampling and remote decision policies under time-decaying information utility. We design two novel algorithms: QuickBLP (a two-stage method) and OnePDSI (a single-layer synchronous iterative method), both theoretically proven to converge at rate O(1/R^K). By reformulating the Bellman equation and applying a Dinkelbach-type transformation, we derive analytical thresholds for sampling gain. Results: Simulations demonstrate that our approach significantly improves decision effectiveness, outperforming AoI-optimal baselines in terms of task-specific utility.

Technology Category

• Machine Learning: Online Learning & Bandits
• Planning, Routing, and Scheduling: Planning with Markov Models (MDPs, POMDPs)
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• User Modeling, Personalization and Recommendation: Fairness-aware retrieval and ranking

Abstract

Data freshness, measured by Age of Information (AoI), is highly relevant in networked applications such as Vehicle to Everything (V2X), smart health systems, and Industrial Internet of Things (IIoT). Yet, freshness alone does not equate to informativeness. In decision-critical settings, some stale data may prove more valuable than fresh updates. To explore this nuance, we move beyond AoI-centric policies and investigate how data staleness impacts decision-making under data-staleness-induced uncertainty. We pose a central question: What is the value of information, when freshness fades, and only its power to shape remote decisions remains? To capture this endured value, we propose AR-MDP, an Age-aware Remote Markov Decision Process framework, which co-designs optimal sampling and remote decision-making under a sampling frequency constraint and random delay. To efficiently solve this problem, we design a new two-stage hierarchical algorithm namely Quick Bellman-Linear-Program (QuickBLP), where the first stage involves solving the Dinkelbach root of a Bellman variant and the second stage involves solving a streamlined linear program (LP). For the tricky first stage, we propose a new One-layer Primal-Dinkelbach Synchronous Iteration (OnePDSI) method, which overcomes the re-convergence and non-expansive divergence present in existing per-sample multi-layer algorithms. Through rigorous convergence analysis of our proposed algorithms, we establish that the worst-case optimality gap in OnePDSI exhibits exponential decay with respect to iteration $K$ at a rate of $mathcal{O}(frac{1}{R^K})$. Through sensitivity analysis, we derive a threshold for the sampling frequency, beyond which additional sampling does not yield further gains in decision-making. Simulation results validate our analyses.

Problem

• Investigates impact of data staleness on remote decision-making effectiveness
• Proposes AR-MDP framework for optimal sampling and decision co-design
• Develops QuickBLP algorithm to solve sampling-constrained decision problems

Innovation

• Proposes AR-MDP for optimal sampling and decision-making
• Introduces QuickBLP two-stage hierarchical algorithm
• Develops OnePDSI method for efficient convergence