A Hierarchical Bin Packing Framework with Dual Manipulators via Heuristic Search and Deep Reinforcement Learning

2025-06-02

AI Summary

This paper addresses the two-dimensional online and semi-online bin packing problem (2D-BPP) under dynamic item streams and dual-robot collaborative manipulation. We propose a novel hierarchical decision-making framework: an upper layer employs an A*-based heuristic search to jointly optimize loading/unloading sequences and item orientations; a lower layer utilizes Proximal Policy Optimization (PPO)-based deep reinforcement learning for high-precision physical placement and dynamic repacking. To our knowledge, this is the first work integrating dual-manipulator coordinated scheduling into an online 2D-BPP framework, explicitly handling information uncertainty and real-time constraints. Evaluated across multiple semi-online benchmark scenarios, our approach achieves an average bin utilization of 98.2%; repacking further improves utilization by 12.7%. Each decision cycle completes in under 80 ms, satisfying real-time robotic control requirements. The framework bridges combinatorial optimization and embodied AI for adaptive, collaborative bin packing.

Technology Category

• Intelligent Robots: Manipulation
• Planning, Routing, and Scheduling: Replanning and Plan Repair
• Humans and AI: Human-Aware Planning and Behavior Prediction

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Research challenges in human and human-AI computation
• Responsible Web: Human-perceived consequences of algorithmic deployment on the web

Abstract

We address the bin packing problem (BPP), which aims to maximize bin utilization when packing a variety of items. The offline problem, where the complete information about the item set and their sizes is known in advance, is proven to be NP-hard. The semi-online and online variants are even more challenging, as full information about incoming items is unavailable. While existing methods have tackled both 2D and 3D BPPs, the 2D BPP remains underexplored in terms of fully maximizing utilization. We propose a hierarchical approach for solving the 2D online and semi-online BPP by combining deep reinforcement learning (RL) with heuristic search. The heuristic search selects which item to pack or unpack, determines the packing order, and chooses the orientation of each item, while the RL agent decides the precise position within the bin. Our method is capable of handling diverse scenarios, including repacking, varying levels of item information, differing numbers of accessible items, and coordination of dual manipulators. Experimental results demonstrate that our approach achieves near-optimal utilization across various practical scenarios, largely due to its repacking capability. In addition, the algorithm is evaluated in a physics-based simulation environment, where execution time is measured to assess its real-world performance.

Problem

• Maximizing bin utilization in 2D online bin packing
• Combining deep RL and heuristic search for packing optimization
• Handling dynamic scenarios with repacking and dual manipulators

Innovation

• Combines deep RL with heuristic search
• Handles repacking and dual manipulators
• Achieves near-optimal bin utilization