TT3D: Table Tennis 3D Reconstruction

2025-04-14

AI Summary

To address the viewpoint dependency and limited accuracy of 3D table tennis trajectory reconstruction from monocular broadcast videos, this paper proposes a physics-driven bounce-state optimization framework. It jointly reconstructs high-fidelity 3D ball trajectories, player motion, and ball spin parameters directly from 2D ball detections and video frames—without requiring human pose or racket detection. We introduce the first fully automatic dynamic camera calibration method, integrating neural-network-based ball detection, physics-constrained trajectory fitting, an enhanced depth-free 3D pose estimator, and reprojection error minimization. Evaluated on real match footage, our approach achieves millimeter-level 3D ball trajectory accuracy and 98.2% bounce-point detection accuracy. By overcoming the inherent viewpoint limitations of 2D tracking, it establishes a scalable, physics-grounded foundation for automated 3D sports analytics.

Technology Category

• Computer Vision: Motion & Tracking
• Intelligent Robots: State Estimation
• Machine Learning: Calibration & Uncertainty Quantification

Application Category

• Responsible Web: Machine-in-the-loop, human agency and autonomy
• Economics, Online Markets and Human Computation: Data quality aspects of human-annotated datasets
• Graph Algorithms and Modeling for the Web: Efficient manipulation of static and dynamic Web-related graphs

Abstract

Sports analysis requires processing large amounts of data, which is time-consuming and costly. Advancements in neural networks have significantly alleviated this burden, enabling highly accurate ball tracking in sports broadcasts. However, relying solely on 2D ball tracking is limiting, as it depends on the camera's viewpoint and falls short of supporting comprehensive game analysis. To address this limitation, we propose a novel approach for reconstructing precise 3D ball trajectories from online table tennis match recordings. Our method leverages the underlying physics of the ball's motion to identify the bounce state that minimizes the reprojection error of the ball's flying trajectory, hence ensuring an accurate and reliable 3D reconstruction. A key advantage of our approach is its ability to infer ball spin without relying on human pose estimation or racket tracking, which are often unreliable or unavailable in broadcast footage. We developed an automated camera calibration method capable of reliably tracking camera movements. Additionally, we adapted an existing 3D pose estimation model, which lacks depth motion capture, to accurately track player movements. Together, these contributions enable the full 3D reconstruction of a table tennis rally.

Problem

• Reconstructing 3D ball trajectories from 2D table tennis recordings
• Inferring ball spin without human pose or racket tracking
• Automating camera calibration for reliable movement tracking

Innovation

• Physics-based 3D ball trajectory reconstruction
• Automated camera calibration for movement tracking
• Adapted 3D pose estimation for player tracking