Technology Category
Computer Vision: Multi-modal VisionMachine Learning: Multimodal LearningNatural Language Processing: Safety and Robustness
RECALL-MM: A Multimodal Dataset of Consumer Product Recalls for Risk Analysis using Computational Methods and Large Language Models
🤖 AI Summary
Early risk identification in product safety design is hindered by data silos and scarce, inconsistently annotated hazard labels. Method: We propose a novel multimodal risk analysis paradigm grounded in historical recall data, introducing RECALL-MM—the first multimodal product safety dataset integrating CPSC textual descriptions with product images. To address label scarcity, we incorporate generative data augmentation and LLM-driven visual risk understanding. Our method features a pioneering “text–image joint embedding + interactive clustering mapping” framework and introduces a zero-shot LLM-based inference paradigm for mapping images directly to hazard categories. Contribution/Results: Experiments demonstrate interpretable, cross-category visualization of recall patterns; the framework achieves accurate prediction of most historical hazard categories using product images alone. This validates the feasibility, generalizability, and engineering utility of data-driven safety-by-design.
Technology Category
Application Category
📝 Abstract
Product recalls provide valuable insights into potential risks and hazards within the engineering design process, yet their full potential remains underutilized. In this study, we curate data from the United States Consumer Product Safety Commission (CPSC) recalls database to develop a multimodal dataset, RECALL-MM, that informs data-driven risk assessment using historical information, and augment it using generative methods. Patterns in the dataset highlight specific areas where improved safety measures could have significant impact. We extend our analysis by demonstrating interactive clustering maps that embed all recalls into a shared latent space based on recall descriptions and product names. Leveraging these data-driven tools, we explore three case studies to demonstrate the dataset's utility in identifying product risks and guiding safer design decisions. The first two case studies illustrate how designers can visualize patterns across recalled products and situate new product ideas within the broader recall landscape to proactively anticipate hazards. In the third case study, we extend our approach by employing a large language model (LLM) to predict potential hazards based solely on product images. This demonstrates the model's ability to leverage visual context to identify risk factors, revealing strong alignment with historical recall data across many hazard categories. However, the analysis also highlights areas where hazard prediction remains challenging, underscoring the importance of risk awareness throughout the design process. Collectively, this work aims to bridge the gap between historical recall data and future product safety, presenting a scalable, data-driven approach to safer engineering design.
Problem
Research questions and friction points this paper is trying to address.
Develops multimodal dataset for data-driven product risk assessment
Identifies safety improvement areas using historical recall patterns
Predicts product hazards using LLMs and visual context
Innovation
Methods, ideas, or system contributions that make the work stand out.
Multimodal dataset for risk assessment
Interactive clustering maps in latent space
LLM predicts hazards from product images
🔎 Similar Papers
No similar papers found.
