Data Fusion of Deep Learned Molecular Embeddings for Property Prediction

2025-04-09

AI Summary

To address the degradation of multi-task learning (MTL) performance in molecular property prediction caused by data sparsity, this paper proposes an embedding-fusion MTL framework. Departing from conventional parameter-sharing paradigms, it innovatively fuses task-specific molecular embeddings—learned independently by GCN- or Transformer-based encoders—at the embedding level. The fused representations are concatenated and dimensionally reduced before undergoing joint regression training. This design alleviates MTL’s reliance on strong inter-task correlations and complete data coverage, thereby substantially improving prediction robustness for weakly correlated and infrequently observed properties. On quantum chemistry benchmark datasets and a custom-built sparse dataset, the method achieves an average 18% reduction in mean absolute error (MAE) over standard MTL baselines, and improves prediction accuracy for data-constrained properties by 23% relative to single-task models.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Computer Vision: Multi-modal Vision
• Multiagent Systems: Multiagent Learning

Application Category

• Graph Algorithms and Modeling for the Web: Graph embeddings and representation learning for Web-related graphs
• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Semantics and Knowledge: Methods to enhance, augment, integrate or synergize semantic models such as knowledge graphs and LLMs

Abstract

Data-driven approaches such as deep learning can result in predictive models for material properties with exceptional accuracy and efficiency. However, in many problems data is sparse, severely limiting their accuracy and applicability. To improve predictions, techniques such as transfer learning and multi-task learning have been used. The performance of multi-task learning models depends on the strength of the underlying correlations between tasks and the completeness of the dataset. We find that standard multi-task models tend to underperform when trained on sparse datasets with weakly correlated properties. To address this gap, we use data fusion techniques to combine the learned molecular embeddings of various single-task models and trained a multi-task model on this combined embedding. We apply this technique to a widely used benchmark dataset of quantum chemistry data for small molecules as well as a newly compiled sparse dataset of experimental data collected from literature and our own quantum chemistry and thermochemical calculations. The results show that the fused, multi-task models outperform standard multi-task models for sparse datasets and can provide enhanced prediction on data-limited properties compared to single-task models.

Problem

• Improving property prediction with sparse data
• Enhancing multi-task learning for weakly correlated properties
• Combining single-task embeddings for better performance

Innovation

• Fuse deep learned molecular embeddings
• Combine single-task model embeddings
• Enhance sparse dataset predictions