OpenTie: Open-vocabulary Sequential Rebar Tying System

2025-08-26

AI Summary

Automated 3D rebar tying in construction sites remains challenging due to reliance on pre-built 3D models and limitations to planar operations. Method: This paper proposes an open-vocabulary, sequence-based rebar-tying system that requires no 3D model training. It integrates open-vocabulary detection with RGB-to-point-cloud generation within a stereo-vision–robotic-arm collaborative framework. Key components include image post-processing, prompt-driven rebar identification, and point-cloud–based spatial localization, enabling end-to-end perception-to-action control. Contribution/Results: The system supports multi-directional (horizontal and vertical) tying and demonstrates high-precision rebar matching and autonomous tying in real-world, unstructured construction environments. It significantly enhances robotic adaptability and deployment efficiency, offering a lightweight, generalizable, and field-deployable solution for on-site rebar operations.

Technology Category

• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Computer Vision: Vision for Robotics & Autonomous Driving
• Planning, Routing, and Scheduling: Optimization of Spatio-temporal Systems

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Robotic practices on the construction site emerge as an attention-attracting manner owing to their capability of tackle complex challenges, especially in the rebar-involved scenarios. Most of existing products and research are mainly focused on flat rebar setting with model training demands. To fulfill this gap, we propose OpenTie, a 3D training-free rebar tying framework utilizing a RGB-to-point-cloud generation and an open-vocabulary detection. We implements the OpenTie via a robotic arm with a binocular camera and guarantees a high accuracy by applying the prompt-based object detection method on the image filtered by our propose post-processing procedure based a image to point cloud generation framework. The system is flexible for horizontal and vertical rebar tying tasks and the experiments on the real-world rebar setting verifies that the effectiveness of the system in practice.

Problem

• Develops open-vocabulary robotic system for rebar tying
• Eliminates need for model training in construction robotics
• Handles both horizontal and vertical rebar configurations

Innovation

• Open-vocabulary detection without training requirements
• RGB-to-point-cloud generation for 3D perception
• Prompt-based object detection with post-processing filtering