An Active Learning-Based Streaming Pipeline for Reduced Data Training of Structure Finding Models in Neutron Diffractometry

2024-12-15

BigData Congress [Services Society]

AI Summary

Neutron diffraction structure determination is computationally expensive, and conventional machine learning approaches face training bottlenecks due to exponential growth in required simulation data. To address this, we propose a streaming active learning framework tailored for neutron scattering: it integrates uncertainty-driven batch-mode sampling with online model updating to dynamically generate high-information simulated samples. The framework jointly incorporates neutron scattering pattern simulation and heterogeneous-platform optimization. Critically, it achieves zero accuracy loss while reducing training data volume by 75% and accelerating end-to-end training by 20%. By significantly alleviating the computational burden of simulating data in high-dimensional structural parameter spaces, our approach establishes a scalable, experiment-driven machine learning paradigm for real-time structural analysis.

Technology Category

• Machine Learning: Structured Learning
• Search and Optimization: Sampling/Simulation-based Search
• Natural Language Processing: Learning & Optimization for NLP

Application Category

• Graph Algorithms and Modeling for the Web: Graph neural networks and deep learning approaches for Web-related graphs
• Web Mining and Content Analysis: Machine learning and data science for the Web
• Systems and Infrastructure for Web, Mobile and WoT: Applied ML and AI for Web-based mobile applications

Abstract

Structure determination workloads in neutron diffractometry are computationally expensive and routinely require several hours to many days to determine the structure of a material from its neutron diffraction patterns. The potential for machine learning models trained on simulated neutron scattering patterns to significantly speed up these tasks have been reported recently. However, the amount of simulated data needed to train these models grows exponentially with the number of structural parameters to be predicted and poses a significant computational challenge. To overcome this challenge, we introduce a novel batch-mode active learning (AL) policy that uses uncertainty sampling to simulate training data drawn from a probability distribution that prefers labelled examples about which the model is least certain. We confirm its efficacy in training the same models with ∼ 75% less training data while improving the accuracy. We then discuss the design of an efficient stream-based training workflow that uses this AL policy and present a performance study on two heterogeneous platforms to demonstrate that, compared with a conventional training workflow, the streaming workflow delivers ∼ 20% shorter training time without any loss of accuracy.

Problem

• Reducing computational cost in neutron diffractometry structure determination
• Minimizing simulated data needed for training machine learning models
• Improving training efficiency without sacrificing model accuracy

Innovation

• Active learning reduces training data by 75%
• Uncertainty sampling optimizes data selection
• Stream-based workflow cuts training time 20%