Multimodal Cyber-physical Interaction in XR: Hybrid Doctoral Thesis Defense

2026-03-16

AI Summary

This study addresses the limitations of traditional doctoral dissertation defenses, which are constrained to in-person settings or two-dimensional video conferencing, resulting in rigid participation formats and insufficient presence. The authors propose a multimodal extended reality (XR) framework that integrates physical attendance, immersive virtual reality (VR), and web browser access to establish a continuous spectrum of hybrid participation. Leveraging full-body motion capture, WebXR standards, and a cross-platform architecture, this work presents the first implementation of a physically-virtually integrated dissertation defense, enabling natural synchronization of user postures and gestures across heterogeneous environments. User studies demonstrate that the system effectively transcends the binary distinction between physical and virtual participation, significantly enhancing presence and interactive experience, thereby validating its feasibility and advantages for diverse hybrid academic events.

Technology Category

• Computer Vision: Multi-modal Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion
• Application Domains: Humanities & Computational Social Science

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Responsible Web: Consent frameworks and practices on the web
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Academic events, such as a doctoral thesis defense, are typically limited to either physical co-location or flat video conferencing, resulting in rigid participation formats and fragmented presence. We present a multimodal framework that breaks this binary by supporting a spectrum of participation - from in-person attendance to immersive virtual reality (VR) or browser access - and report our findings from using it to organize the first ever hybrid doctoral thesis defense using extended reality (XR). The framework integrates full-body motion tracking to synchronize the user's avatar motions and gestures, enabling natural interaction with onsite participants as well as body language and gestures with remote attendees in the virtual world. It leverages WebXR to provide cross-platform and instant accessibility with easy setup. User feedback analysis reveals positive VR experiences and demonstrates the framework's effectiveness in supporting various hybrid event activities.

Problem

• Multimodal Interaction
• Cyber-physical Systems
• Extended Reality
• Hybrid Events
• Doctoral Thesis Defense

Innovation

• Multimodal Interaction
• Extended Reality (XR)
• Hybrid Events
• Full-body Motion Tracking
• WebXR