Building Workflows for Interactive Human in the Loop Automated Experiment (hAE) in STEM-EELS

2024-04-10

arXiv.org

AI Summary

In STEM-EELS, sparse critical structures—such as atomic-scale defects—are difficult to identify efficiently and are prone to suboptimal local attention. Method: This paper proposes a human-in-the-loop adaptive experimentation (hAE) framework. It systematically characterizes, for the first time, how deep kernel learning (DKL) hyperparameters influence Bayesian optimization exploration trajectories; introduces a dual-track monitoring mechanism—operating concurrently in feature space and real space—to enable expert real-time intervention; and integrates local structural descriptors, multi-scale spectro-spatial joint modeling, and real-time knowledge-state assessment. Contribution/Results: The framework breaks from conventional uniform sampling paradigms, significantly improving discovery efficiency of target structures while avoiding local optima. It is empirically validated on EELS experiments and generalizable to other spectroscopic imaging techniques, including 4D-STEM.

Technology Category

• Search and Optimization: Learning to Search
• Machine Learning: Kernel Methods
• Humans and AI: Human-in-the-loop Machine Learning

Application Category

• Semantics and Knowledge: Data modeling to support human-machine intelligence, including LLMs agents, intelligent system behavior, explanations, and user-friendly interactions
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Economics, Online Markets and Human Computation: Humans versus LLMs for data annotation and labeling

Abstract

Exploring the structural, chemical, and physical properties of matter on the nano- and atomic scales has become possible with the recent advances in aberration-corrected electron energy-loss spectroscopy (EELS) in scanning transmission electron microscopy (STEM). However, the current paradigm of STEM-EELS relies on the classical rectangular grid sampling, in which all surface regions are assumed to be of equal a priori interest. This is typically not the case for real-world scenarios, where phenomena of interest are concentrated in a small number of spatial locations. One of foundational problems is the discovery of nanometer- or atomic scale structures having specific signatures in EELS spectra. Here we systematically explore the hyperparameters controlling deep kernel learning (DKL) discovery workflows for STEM-EELS and identify the role of the local structural descriptors and acquisition functions on the experiment progression. In agreement with actual experiment, we observe that for certain parameter combinations the experiment path can be trapped in the local minima. We demonstrate the approaches for monitoring automated experiment in the real and feature space of the system and monitor knowledge acquisition of the DKL model. Based on these, we construct intervention strategies, thus defining human-in the loop automated experiment (hAE). This approach can be further extended to other techniques including 4D STEM and other forms of spectroscopic imaging.

Problem

• STEM-EELS
• Automated Experimental Design
• Nanoscopic Structure Analysis

Innovation

• Deep Kernel Learning
• Automated Experiment Design
• Nanoscopic and Atomic Structure Analysis