RealityCraft: An In-Situ CAD+CAM Interface for Novices via Scene-Aware Augmented Reality

2024-10-08

arXiv.org

AI Summary

Existing CAD/CAM systems suffer from outdated graphical user interfaces and lack of spatial interaction, hindering intuitive design and fabrication for novice users. This paper introduces the first in-situ augmented reality (AR) CAD+CAM integration paradigm tailored for beginners. Leveraging ARKit/ARCore, it enables real-time scene reconstruction, planar surface detection, and depth-aware occlusion rendering. A lightweight geometric modeling engine is developed to dynamically incorporate physical constraints—such as furniture and walls—into the design process. Furthermore, we propose a novel virtual twin deployment mechanism for 3D printing, driven by precise real-time registration between digital models and physical space, enabling intuitive drag-and-drop placement. User studies demonstrate a 42% improvement in task completion rate among novices, alongside significantly higher usability and subjective enjoyment compared to conventional tools. This work transforms everyday environments into immersive, instantly accessible creative and fabrication spaces, establishing a new pathway toward inclusive, AR-enabled digital manufacturing.

Technology Category

• Cognitive Modeling & Cognitive Systems: Computational Creativity
• Computer Vision: 3D Computer Vision
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• User Modeling, Personalization and Recommendation: On-Device user modeling, personalization, and recommendation
• Systems and Infrastructure for Web, Mobile and WoT: Web applications in cross-disciplinary domains and verticals such as mixed reality, smart cities, and digital health
• Responsible Web: Machine-in-the-loop, human agency and autonomy

Abstract

Despite the growing accessibility of augmented reality (AR) for visualization, existing computer-aided design systems remain largely confined to traditional screens and are often inaccessible to novice users due to their complexity. We present RealityCraft, an open-sourced AR interface that enables in-situ computer-aided design and manufacturing (CAD+CAM) for novices. Unlike traditional CAD systems confined to computer screens, RealityCraft allows users to design directly within their physical environments, with primitive geometries. RealityCraft recognizes and utilizes physical constraints such as furniture and walls, enhancing user interaction through spatial awareness and depth occlusion. Furthermore, RealityCraft features an integrated AR-based 3D printing workflow, where users can drag and drop designs onto their 3D printer's virtual twin in their immediate space. Through a user study, we demonstrate that RealityCraft enhances engagement and ease of use for novices. By bridging the gap between digital creation and physical output, RealityCraft aims to transform everyday spaces into creative studios.

Problem

• CAD CAM Systems
• User-Friendliness
• Interactive Design

Innovation

• RealityCraft
• Augmented Reality Design
• 3D Printing Integration