Exploring Cross-Reality Transitions between Projections and Head-Mounted Displays for Immersive Digital Art

2026-08-05

AI Summary

This study addresses the unclear perceptual mechanisms underlying cross-reality transitions in heterogeneous immersive environments composed of large-scale projections and mixed reality headsets. The authors construct a hybrid art installation integrating projection, augmented reality, and virtual reality to investigate, through controlled experiments, how object-level and scene-level transitions affect user experience. They innovatively introduce “controllable inconsistency” as a diagnostic probe, revealing that spatial misalignment, visual appearance discrepancies, and latency respectively impair action predictability, aesthetic coherence, and sense of responsiveness. Notably, digital assets—such as static models, skeletal animations, and particle systems—exhibit markedly different sensitivities to these inconsistencies. Based on these findings, the work advocates for asset-type-specific design strategies. The system, built on Unreal Engine, incorporates spatial calibration, visual synchronization, and cross-device latency control, and includes the open-source plugin HUICRSync to support projection–MR prototyping.

Abstract

Immersive exhibitions increasingly combine large-scale projections with mixed reality (MR) head-mounted displays (HMDs), but how users perceive transitions across these heterogeneous visualization environments remains underexplored. We investigate cross-reality (CR) object- and scene-level transitions in a hybrid immersive art installation spanning projection, augmented reality (AR), and virtual reality (VR). In a within-subjects study (N=24), we compared a calibrated condition with a bundled inconsistency condition introducing noticeable differences in spatial alignment, visual appearance, and cross-device latency. This contrast was used not to establish perceptual thresholds or hardware targets, but as a diagnostic probe to make transition disruptions more perceptible and discussable. The inconsistency condition reduced presence and increased workload. Interviews showed that spatial misalignment disrupted action-outcome predictability, appearance mismatch reduced aesthetic coherence, and latency weakened perceived responsiveness. Effects also varied by asset type: rigid static assets exposed inconsistencies more clearly, animated skeletal assets shifted attention toward motion, and particle effects often masked small discrepancies. These findings clarify how users perceive projection-MR transitions and support asset-aware design strategies for coherent hybrid immersive art. We also release HUICRSync, an open-source Unreal Engine plugin for prototyping projection-MR CR experiences.

Problem

• cross-reality transitions
• immersive digital art
• projection-MR integration
• user perception
• heterogeneous visualization environments

Innovation

• Cross-Reality
• Immersive Art
• Projection-MR Transitions
• Asset-Aware Design
• HUICRSync