🤖 AI Summary
Existing object detection models lack robustness evaluation against semantic-level perturbations in safety-critical scenarios, as conventional pixel-level perturbations fail to capture real-world semantic variations. To address this gap, this work proposes SemProbe—the first framework integrating controllable diffusion inpainting with safety assessment—enabling users to interactively design domain-relevant semantic perturbations via masking, automatically generate inpainted samples, and quantify model performance degradation through inference. The system supports batch experimentation, parallel workflow management, and structured logging to ensure traceable semantic robustness validation. Evaluated on a circular saw hand-detection task under insurance-guided testing criteria, SemProbe successfully generates effective semantic probes that expose critical model vulnerabilities under safety-relevant conditions.
📝 Abstract
Testing object detectors in safety-critical domains requires semantically meaningful probes beyond pixel-level corruptions. We present SemProbe, a tool for semantic robustness probing: users upload deployment images, create masks manually or automatically, select operational design domain-derived factors (or custom prompts), and run diffusion-based controlled inpainting. The system supports batch jobs, parallel seed/workflow variations, and configurable generation parameters. After each output, model inference runs automatically and displays annotated before/after comparisons with performance deltas. All probes are logged as structured artifacts, enabling traceable robustness evidence aligned with safety evaluation workflows. We demonstrate \textsc{SemProbe} on hand detection for dimension saws, targeting factors from insurance-oriented test criteria.