Integrating AIs With Body Tracking Technology for Human Behaviour Analysis: Challenges and Opportunities

2025-06-24

AI Summary

This work addresses key challenges in wearable-free human behavior analysis—namely, poor coordination among AI components and insufficient engineering of tracking pipelines—by proposing a multi-AI-module collaborative posture-tracking architecture. Methodologically, it integrates off-the-shelf depth cameras, pre-trained AI recognition models, and machine learning–driven non-contact pose estimation into an end-edge-cloud distributed tracking pipeline, deployed within a large-screen remote collaboration system. Key contributions include: (1) dynamic orchestration and low-coupling integration of heterogeneous AI modules; (2) real-time behavioral sensing and retrospective analysis across geographically distributed wall-mounted displays; and (3) empirical validation of a highly scalable, low-barrier AI-augmented interactive system. Experiments conducted in realistic remote collaboration settings demonstrate strong robustness, scalability, and operational feasibility.

Technology Category

• Humans and AI: Human-Aware Planning and Behavior Prediction
• Computer Vision: Remote Sensing / Geospatial AI
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

The automated analysis of human behaviour provides many opportunities for the creation of interactive systems and the post-experiment investigations for user studies. Commodity depth cameras offer reasonable body tracking accuracy at a low price point, without the need for users to wear or hold any extra equipment. The resulting systems typically perform body tracking through a dedicated machine learning model, but they can be enhanced with additional AI components providing extra capabilities. This leads to opportunities but also challenges, for example regarding the orchestration of such AI components and the engineering of the resulting tracking pipeline. In this paper, we discuss these elements, based on our experience with the creation of a remote collaboration system across distant wall-sized displays, that we built using existing and readily available building blocks, including AI-based recognition models.

Problem

• Enhancing human behavior analysis with AI and body tracking
• Integrating multiple AI components for improved tracking pipelines
• Addressing challenges in remote collaboration using AI recognition models

Innovation

• Integrates AI with body tracking technology
• Uses commodity depth cameras for tracking
• Enhances tracking with additional AI components