BASKET: A Large-Scale Video Dataset for Fine-Grained Skill Estimation

2025-03-26

AI Summary

This work addresses the challenge of fine-grained motor skill assessment in long-duration, high-resolution basketball videos. We introduce BASKET, the largest publicly available basketball video dataset to date (4,477 hours, covering 32,232 players), featuring the first cross-gender, cross-age, cross-regional, and cross-proficiency fine-grained annotations for 20 distinct skills (e.g., dribbling control, shooting stability). We formulate a novel long-video-driven multi-skill joint assessment task and propose a weakly supervised learning framework integrating multi-scale spatiotemporal modeling, action-semantic alignment, and hierarchical skill classification. Experiments reveal that state-of-the-art video foundation models significantly underperform human evaluators on this task. To foster research and real-world deployment—particularly in talent scouting and personalized training—we release the dataset, benchmark, and code.

Technology Category

• Computer Vision: Video Understanding & Activity Analysis
• Machine Learning: Multimodal Learning
• Knowledge Representation and Reasoning: Geometric, Spatial, and Temporal Reasoning

Application Category

• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking

Abstract

We present BASKET, a large-scale basketball video dataset for fine-grained skill estimation. BASKET contains 4,477 hours of video capturing 32,232 basketball players from all over the world. Compared to prior skill estimation datasets, our dataset includes a massive number of skilled participants with unprecedented diversity in terms of gender, age, skill level, geographical location, etc. BASKET includes 20 fine-grained basketball skills, challenging modern video recognition models to capture the intricate nuances of player skill through in-depth video analysis. Given a long highlight video (8-10 minutes) of a particular player, the model needs to predict the skill level (e.g., excellent, good, average, fair, poor) for each of the 20 basketball skills. Our empirical analysis reveals that the current state-of-the-art video models struggle with this task, significantly lagging behind the human baseline. We believe that BASKET could be a useful resource for developing new video models with advanced long-range, fine-grained recognition capabilities. In addition, we hope that our dataset will be useful for domain-specific applications such as fair basketball scouting, personalized player development, and many others. Dataset and code are available at https://github.com/yulupan00/BASKET.

Problem

• Estimating fine-grained basketball skills from long videos
• Addressing diversity in player gender, age, and skill levels
• Improving video models for nuanced skill recognition

Innovation

• Large-scale basketball video dataset
• Fine-grained skill estimation model
• Advanced long-range recognition capabilities