A real-time RGB-D perception pipeline for autonomous impact hammers in mining: self-filtering, rock segmentation and rock-breaking poses generation

2026-07-22

AI Summary

This work addresses the limitations of teleoperated percussive hammers in underground mining, which lack real-time perception and autonomous operation capabilities. The authors propose a real-time RGB-D perception pipeline tailored for mining percussive hammers, integrating image-based instance segmentation with point cloud geometric processing to achieve robot-free 3D reconstruction of the workspace, instance-level rock segmentation, and generation of feasible breaking poses—all implemented on an embedded platform. The system operates at 10 Hz with an end-to-end latency of approximately 675 ms. Experimental validation in a scaled physical environment demonstrates that the pipeline meets the real-time requirements for autonomous hammering tasks and exhibits strong potential for practical deployment in mining operations.

Abstract

Impact hammers, also known as rock-breakers, are essential machines in mining operations, where they perform secondary reduction. In underground mining, these machines are typically teleoperated, limiting operational efficiency. This paper presents a real-time RGB-D perception pipeline as a step towards automating the operation of hydraulic impact hammers used in mining. The proposed system simultaneously generates operationally feasible rock-breaking poses and a robot-free 3D representation of the workspace. The proposed approach combines image-based instance segmentation with geometric point cloud processing, and operates on embedded hardware at approximately 10 Hz with a total latency of around 675 ms, enabling responsive closed-loop behavior when integrated with a control system. Experimental results in a representative scaled scenario demonstrate that the proposed system is suitable for real-time autonomous impact hammer operation.

Problem

• autonomous impact hammers
• rock segmentation
• real-time perception
• robot-free workspace representation
• rock-breaking poses

Innovation

• RGB-D perception
• rock segmentation
• pose generation
• real-time autonomy
• embedded system