The Way Up: A Dataset for Hold Usage Detection in Sport Climbing

2025-05-19

AI Summary

A lack of fine-grained, annotated datasets for competitive sport climbing—particularly lacking ground-truth labels for hold contact locations, usage sequence, and temporal stamps—severely hinders motion analysis and AI-assisted training. Method: We introduce the first fine-grained dataset of hold usage in competitive climbing (22 videos) and establish the first benchmark for hold usage detection. We identify core challenges in 2D pose estimation for this domain, including severe limb occlusion and matching of small-scale holds. To address these, we propose a keypoint trajectory modeling approach based on HRNet/HigherHRNet, integrating joint motion trajectories with spatial overlap analysis between keypoints and hold regions to infer grip events. Contribution/Results: We conduct systematic evaluation of state-of-the-art models on our dataset, quantitatively measuring hold usage recognition performance. This work provides the first reproducible benchmark and technical paradigm for AI-driven sport climbing research.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: Manipulation
• Planning, Routing, and Scheduling: Activity and Plan Recognition

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Algorithms and analysis for incomplete, noisy, or partially observed Web-related graphs
• Web Mining and Content Analysis: Large pretrained models with web data

Abstract

Detecting an athlete's position on a route and identifying hold usage are crucial in various climbing-related applications. However, no climbing dataset with detailed hold usage annotations exists to our knowledge. To address this issue, we introduce a dataset of 22 annotated climbing videos, providing ground-truth labels for hold locations, usage order, and time of use. Furthermore, we explore the application of keypoint-based 2D pose-estimation models for detecting hold usage in sport climbing. We determine usage by analyzing the key points of certain joints and the corresponding overlap with climbing holds. We evaluate multiple state-of-the-art models and analyze their accuracy on our dataset, identifying and highlighting climbing-specific challenges. Our dataset and results highlight key challenges in climbing-specific pose estimation and establish a foundation for future research toward AI-assisted systems for sports climbing.

Problem

• Lack of annotated climbing dataset for hold usage detection
• Detecting hold usage via keypoint-based 2D pose estimation
• Evaluating pose-estimation models for climbing-specific challenges

Innovation

• Introduces annotated climbing videos dataset
• Uses keypoint-based 2D pose-estimation models
• Analyzes joint keypoints and hold overlap