Prior Availability in Industrial Visual Sim-to-Real: A Review of CAD-Guided and CAD-Unavailable Regimes

2026-05-28

AI Summary

This work addresses the challenge of sim-to-real transfer in industrial visual inspection, where multiple domain gaps arise from discrepancies in sensors, lighting conditions, materials, and defect patterns. The authors propose a unified framework centered on the availability of prior knowledge, systematically integrating three scenarios: CAD-available, CAD-unavailable, and boundary cases, thereby harmonizing CAD-driven pose estimation and CAD-free anomaly detection paradigms. Their approach combines CAD-based rendering, RGB-D simulation, synthetic defect generation, pre-trained features, vision-language priors, and test-time geometric consistency verification. Experiments on T-LESS/BOP, MVTec AD, and VisA benchmarks demonstrate that transfer performance hinges more critically on source distribution design, detector capacity, and minimal real-world calibration than on the quantity of CAD renderings; notably, CAD models at test time effectively enable mask generation, pose refinement, and depth consistency validation.

Abstract

Industrial visual sim-to-real is often described as transferring from synthetic images to real images, but industrial deployment usually involves a broader mismatch between available evidence and required decisions. A system may be built from CAD renderings, simulated RGB-D observations, normal reference images, synthetic defects, pretrained feature spaces, or language prompts, yet deployed under different sensors, lighting, materials, fixtures, calibration, production variation, and rare defect modes. This review reframes industrial visual sim-to-real as a domain-gap problem organized by prior availability. We distinguish CAD-available settings, where explicit object geometry can support rendering, calibration, pose estimation, segmentation, and test-time geometric verification; CAD-unavailable settings, where geometry is replaced by normal-reference appearance, feature distributions, teacher-student residuals, synthetic anomaly assumptions, foundation features, or vision-language priors; and boundary-prior settings, where approximate models, templates, reference views, or semantic correspondences preserve only part of the CAD role. This framing connects CAD-based detection and 6D pose-estimation literature with industrial anomaly and surface-inspection literature that is usually reviewed separately. To make the taxonomy concrete, we use empirical anchors on T-LESS/BOP, MVTec AD, and VisA. The anchors show that CAD render count alone does not close transfer; source-distribution design, detector capacity, and small real calibration can matter more. They also show that CAD at test time creates a distinct verification channel through mask, pose, and depth consistency, whereas CAD-unavailable inspection relies on calibrated normality and feature deviation. The review therefore argues against a single cross-task leaderboard and instead asks what prior grounds the deployment decision.

Problem

• sim-to-real
• domain gap
• CAD availability
• industrial visual inspection
• prior information

Innovation

• sim-to-real
• CAD availability
• domain gap
• industrial visual inspection
• prior-guided transfer