DR-PETS: Learning-Based Control With Planning in Adversarial Environments

2025-03-26

AI Summary

To address the lack of robustness against adversarial parameter perturbations in real-world decision-making, this paper proposes Distributionally Robust Probabilistic Ensembles with Trajectory Sampling (DR-PETS). The method pioneers the integration of distributionally robust optimization into the PETS model-predictive control paradigm, employing a $p$-Wasserstein ambiguity set to characterize model uncertainty and embedding a tractable convex min-max optimization within trajectory planning to proactively guarantee robustness against worst-case perturbations. Unlike standard PETS—which passively handles stochasticity—DR-PETS provides theoretically grounded adversarial robustness. Empirical evaluation on the inverted pendulum and cart-pole tasks under strong adversarial disturbances demonstrates that DR-PETS maintains a control success rate above 92%, substantially outperforming standard PETS (which achieves less than 40%). These results validate both the effectiveness and practical applicability of the proposed framework.

Technology Category

• Machine Learning: Adversarial Learning & Robustness
• Multiagent Systems: Adversarial Agents
• Intelligent Robots: State Estimation

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

Ensuring robustness against epistemic, possibly adversarial, perturbations is essential for reliable real-world decision-making. While the Probabilistic Ensembles with Trajectory Sampling (PETS) algorithm inherently handles uncertainty via ensemble-based probabilistic models, it lacks guarantees against structured adversarial or worst-case uncertainty distributions. To address this, we propose DR-PETS, a distributionally robust extension of PETS that certifies robustness against adversarial perturbations. We formalize uncertainty via a p-Wasserstein ambiguity set, enabling worst-case-aware planning through a min-max optimization framework. While PETS passively accounts for stochasticity, DR-PETS actively optimizes robustness via a tractable convex approximation integrated into PETS planning loop. Experiments on pendulum stabilization and cart-pole balancing show that DR-PETS certifies robustness against adversarial parameter perturbations, achieving consistent performance in worst-case scenarios where PETS deteriorates.

Problem

• Ensures robustness against adversarial perturbations in decision-making
• Extends PETS to handle worst-case uncertainty distributions
• Certifies performance in adversarial scenarios via min-max optimization

Innovation

• Distributionally robust extension of PETS
• Uses p-Wasserstein ambiguity set
• Integrates convex approximation in planning