Multimodal learning of melt pool dynamics in laser powder bed fusion

2025-09-03

AI Summary

To address the reliance of laser powder bed fusion (LPBF) melt pool monitoring on costly high-speed X-ray imaging, this work proposes a multimodal fusion-driven transfer learning framework. Methodologically, it integrates high spatiotemporal-resolution X-ray images with low-cost photodiode absorptivity signals, employing a CNN-RNN hybrid architecture to jointly model spatial and temporal features. Early fusion and knowledge distillation are leveraged to transfer knowledge from the multimodal model to a lightweight unimodal model that uses only absorptivity as input. Key contributions include: (i) the first demonstration of high-accuracy prediction of melt pool geometry and dynamics using absorptivity data alone; (ii) substantial reduction in dependence on X-ray equipment, enabling industrial-grade real-time closed-loop monitoring; and (iii) a 32.7% reduction in prediction error compared to conventional unimodal approaches, while maintaining model compactness and computational efficiency.

Technology Category

• Computer Vision: Multi-modal Vision
• Machine Learning: Multimodal Learning
• Intelligent Robots: Multimodal Perception & Sensor Fusion

Application Category

• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs
• Search and Retrieval-Augmented AI: Retrieval-Augmented Generation (RAG) and multi-modal RAG

Abstract

While multiple sensors are used for real-time monitoring in additive manufacturing, not all provide practical or reliable process insights. For example, high-speed X-ray imaging offers valuable spatial information about subsurface melt pool behavior but is costly and impractical for most industrial settings. In contrast, absorptivity data from low-cost photodiodes correlate with melt pool dynamics but is often too noisy for accurate prediction when used alone. In this paper, we propose a multimodal data fusion approach for predicting melt pool dynamics by combining high-fidelity X-ray data with low-fidelity absorptivity data in the Laser Powder Bed Fusion (LPBF) process. Our multimodal learning framework integrates convolutional neural networks (CNNs) for spatial feature extraction from X-ray data with recurrent neural networks (RNNs) for temporal feature extraction from absorptivity signals, using an early fusion strategy. The multimodal model is further used as a transfer learning model to fine-tune the RNN model that can predict melt pool dynamics only with absorptivity, with greater accuracy compared to the multimodal model. Results show that training with both modalities significantly improves prediction accuracy compared to using either modality alone. Furthermore, once trained, the model can infer melt pool characteristics using only absorptivity data, eliminating the need for expensive X-ray imaging. This multimodal fusion approach enables cost-effective, real-time monitoring and has broad applicability in additive manufacturing.

Problem

• Predicting melt pool dynamics using multimodal sensor fusion
• Combining high-fidelity X-ray with low-fidelity absorptivity data
• Enabling cost-effective real-time monitoring in additive manufacturing

Innovation

• Multimodal fusion of X-ray and absorptivity data
• CNN-RNN architecture for spatiotemporal feature extraction
• Transfer learning enables single-sensor inference capability