Traversing Dual Realities: Investigating Techniques for Transitioning 3D Objects between Desktop and Augmented Reality Environments

2025-04-01

AI Summary

This study addresses the challenge of seamless 3D object migration and collaborative manipulation between desktop and augmented reality (AR) environments. We propose and systematically evaluate three cross-platform 3D object transition techniques, establishing— for the first time—a desktop+AR collaborative interaction paradigm grounded in authentic scientific workflows (e.g., computational chemistry). Using Unity and the Mixed Reality Toolkit (MRTK), we implement a hybrid environment integrating spatial anchoring, gesture mapping, and context-aware state synchronization to ensure cross-domain persistence and continuous 3D interaction. A user study with 24 participants demonstrates a 32% improvement in expert task completion efficiency and a 41% reduction in error rate. The work distills reusable design principles and provides both methodological guidance and empirical validation for cross-reality 3D workflows.

Technology Category

• Humans and AI: Interaction Techniques and Devices
• Computer Vision: 3D Computer Vision
• Cognitive Modeling & Cognitive Systems: Social Cognition And Interaction

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Economics, Online Markets and Human Computation: Trust and reliance of crowd workers and data experts on GenAI
• Search and Retrieval-Augmented AI: Assisted, interactive, and conversational search

Abstract

Desktop environments can integrate augmented reality (AR) head-worn devices to support 3D representations, visualizations, and interactions in a novel yet familiar setting. As users navigate across the dual realities -- desktop and AR -- a way to move 3D objects between them is needed. We devise three baseline transition techniques based on common approaches in the literature and evaluate their usability and practicality in an initial user study (N=18). After refining both our transition techniques and the surrounding technical setup, we validate the applicability of the overall concept for real-world activities in an expert user study (N=6). In it, computational chemists followed their usual desktop workflows to build, manipulate, and analyze 3D molecular structures, but now aided with the addition of AR and our transition techniques. Based on our findings from both user studies, we provide lessons learned and takeaways for the design of 3D object transition techniques in desktop + AR environments.

Problem

• Develop methods to move 3D objects between desktop and AR environments
• Evaluate usability of transition techniques in user studies
• Apply AR transitions to real-world tasks like molecular modeling

Innovation

• Integrating AR head-worn devices with desktop environments
• Developing three baseline 3D object transition techniques
• Validating concept with computational chemists' workflows