Real-time 3D Light-field Viewing with Eye-tracking on Conventional Displays

2025-08-22

AI Summary

Conventional 3D light-field visualization relies on expensive, specialized hardware—such as VR headsets or active shutter glasses—posing high accessibility barriers and limiting widespread adoption. Method: This paper proposes a low-cost, real-time light-field viewing system requiring only a standard 2D display, an RGB camera, and anaglyph (red-cyan) glasses. It integrates lightweight CPU-based eye tracking, dynamic selection of pre-captured light-field views, and real-time stereo synthesis—all executed entirely on CPU without GPU acceleration or custom hardware. Contribution/Results: To our knowledge, this is the first CPU-only system enabling real-time, interactive, eye-tracking-driven light-field rendering. It achieves stable operation at 30 FPS, significantly enhancing immersion and accessibility. The approach establishes a deployable, low-barrier 3D visualization paradigm suitable for resource-constrained environments—including education and digital media—without compromising visual fidelity or interactivity.

Technology Category

• Computer Vision: 3D Computer Vision
• Machine Learning: Hardware-aware ML
• Humans and AI: Interaction Techniques and Devices

Application Category

• Economics, Online Markets and Human Computation: Cost models of using LLMs in production systems
• Security and Privacy: Large-scale security measurements
• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation

Abstract

Creating immersive 3D visual experiences typically requires expensive and specialized hardware such as VR headsets, autostereoscopic displays, or active shutter glasses. These constraints limit the accessibility and everyday use of 3D visualization technologies in resource-constrained settings. To address this, we propose a low-cost system that enables real-time 3D light-field viewing using only a standard 2D monitor, a conventional RGB webcam, and red-cyan anaglyph glasses. The system integrates real-time eye-tracking to dynamically adapt the displayed light-field image to the user's head position with a lightweight rendering pipeline that selects and composites stereoscopic views from pre-captured light-field data. The resulting anaglyph image is updated in real-time, creating a more immersive and responsive 3D experience. The system operates entirely on CPU and maintains a stable frame rate of 30 FPS, confirming its feasibility on typical consumer-grade hardware. All of these highlight the potential of our approach as an accessible platform for interactive 3D applications in education, digital media, and beyond.

Problem

• Enabling immersive 3D viewing without expensive specialized hardware
• Adapting light-field images to user's head position in real-time
• Providing accessible 3D visualization for resource-constrained settings

Innovation

• Uses standard monitor and webcam
• Real-time eye-tracking for adaptive rendering
• Lightweight CPU-based 3D light-field processing