Confidence Adjusted Surprise Measure for Active Resourceful Trials (CA-SMART): A Data-driven Active Learning Framework for Accelerating Material Discovery under Resource Constraints

2025-03-27

AI Summary

To address resource constraints in materials discovery—including high experimental costs, vast search spaces, and time-consuming property representation—this paper proposes a data-driven active learning framework guided by “surprisal.” The core contribution is the Confidence-Adjusted Surprisal (CAS) metric, the first of its kind, which dynamically modulates exploration intensity across high- and low-confidence regions of the surrogate model to adaptively balance exploration and exploitation. The method integrates Bayesian active learning, uncertainty quantification from surrogate models, and confidence-weighted surprisal modeling. Evaluated on benchmark functions (Six-Hump Camelback and Griewank) and a real-world steel fatigue strength prediction task, the framework achieves significant improvements in both predictive accuracy and sampling efficiency over conventional surprisal metrics, standard Bayesian optimization, and baseline machine learning approaches.

Technology Category

• Machine Learning: Active Learning
• Search and Optimization: Sampling/Simulation-based Search
• Constraint Satisfaction and Optimization: Constraint Learning and Acquisition

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Representation, reconstruction, and subgraph or motif discovery in Web-related graphs
• User Modeling, Personalization and Recommendation: Attacks and countermeasures in recommendation systems

Abstract

Accelerating the discovery and manufacturing of advanced materials with specific properties is a critical yet formidable challenge due to vast search space, high costs of experiments, and time-intensive nature of material characterization. In recent years, active learning, where a surrogate machine learning (ML) model mimics the scientific discovery process of a human scientist, has emerged as a promising approach to address these challenges by guiding experimentation toward high-value outcomes with a limited budget. Among the diverse active learning philosophies, the concept of surprise (capturing the divergence between expected and observed outcomes) has demonstrated significant potential to drive experimental trials and refine predictive models. Scientific discovery often stems from surprise thereby making it a natural driver to guide the search process. Despite its promise, prior studies leveraging surprise metrics such as Shannon and Bayesian surprise lack mechanisms to account for prior confidence, leading to excessive exploration of uncertain regions that may not yield useful information. To address this, we propose the Confidence-Adjusted Surprise Measure for Active Resourceful Trials (CA-SMART), a novel Bayesian active learning framework tailored for optimizing data-driven experimentation. On a high level, CA-SMART incorporates Confidence-Adjusted Surprise (CAS) to dynamically balance exploration and exploitation by amplifying surprises in regions where the model is more certain while discounting them in highly uncertain areas. We evaluated CA-SMART on two benchmark functions (Six-Hump Camelback and Griewank) and in predicting the fatigue strength of steel. The results demonstrate superior accuracy and efficiency compared to traditional surprise metrics, standard Bayesian Optimization (BO) acquisition functions and conventional ML methods.

Problem

• Accelerating material discovery under resource constraints.
• Improving active learning by adjusting surprise with confidence.
• Balancing exploration and exploitation in data-driven experimentation.

Innovation

• Bayesian active learning framework
• Confidence-Adjusted Surprise (CAS)
• dynamic exploration-exploitation balance