Perceptive Hierarchical-Task MPC for Sequential Mobile Manipulation in Unstructured Semi-Static Environments

2026-03-10

AI Summary

This work addresses the challenge of sequential mobile manipulation tasks in unstructured, semi-static environments, where traditional approaches relying on static maps often fail due to dynamic changes such as object addition, removal, or displacement. To overcome this limitation, the authors propose a perception-aware Hierarchical Task Model Predictive Control (HTMPC) framework that, for the first time, integrates Bayesian explicit modeling of object-level environmental changes with hierarchical task MPC. The method maintains an accurate, real-time 3D representation of the environment without requiring prior maps or external infrastructure and employs lexicographic optimization to efficiently generate whole-body motion policies. Experimental results in both simulation and real-world robotic platforms demonstrate significant improvements over baseline methods, showing robust handling of moving and phantom obstacles and substantially enhanced task completion efficiency and environmental adaptability.

Technology Category

• Intelligent Robots: Manipulation
• Humans and AI: Human-Aware Planning and Behavior Prediction
• Reasoning under Uncertainty: Sequential Decision Making

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Energy management for devices in mobile Web and WoT environments
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

As compared to typical mobile manipulation tasks, sequential mobile manipulation poses a unique challenge -- as the robot operates over extended periods, successful task completion is not solely dependent on consistent motion generation but also on the robot's awareness and adaptivity to changes in the operating environment. While existing motion planners can generate whole-body trajectories to complete sequential tasks, they typically assume that the environment remains static and rely on precomputed maps. This assumption often breaks down during long-term operations, where semi-static changes such as object removal, introduction, or shifts are common. In this work, we propose a novel perceptive hierarchical-task model predictive control (HTMPC) framework for efficient sequential mobile manipulation in unstructured, changing environments. To tackle the challenge, we leverage a Bayesian inference framework to explicitly model object-level changes and thereby maintain a temporally accurate representation of the 3D environment; this up-to-date representation is embedded in a lexicographic optimization framework to enable efficient execution of sequential tasks. We validate our perceptive HTMPC approach through both simulated and real-robot experiments. In contrast to baseline methods, our approach systematically accounts for moved and phantom obstacles, successfully completing sequential tasks with higher efficiency and reactivity, without relying on prior maps or external infrastructure.

Problem

• sequential mobile manipulation
• semi-static environments
• environmental changes
• motion planning
• mobile robotics

Innovation

• Perceptive MPC
• Hierarchical-Task MPC
• Bayesian inference
• semi-static environments
• mobile manipulation