A Strong Baseline for Molecular Few-Shot Learning

2024-04-02

AI Summary

This work addresses the robustness of molecular few-shot learning under domain shift in drug discovery, challenging mainstream meta-learning paradigms by proposing a lightweight, fine-tuning-based framework. Methodologically, it introduces (1) a regularized quadratic probe loss incorporating Mahalanobis distance into few-shot fine-tuning to enhance intra-class compactness and inter-class discriminability; (2) a dedicated block-coordinate descent optimizer designed to avoid degenerate solutions; and (3) the first few-shot benchmark specifically curated for evaluating domain-shift robustness in molecular representation learning. The method is model-agnostic—compatible with any molecular graph neural network—and supports plug-and-play integration of black-box models without episodic pretraining. Experiments demonstrate competitive performance against state-of-the-art meta-learning methods on standard few-shot tasks, while significantly outperforming them under domain shift.

Technology Category

• Machine Learning: Transfer, Domain Adaptation, Multi-Task Learning
• Search and Optimization: Learning to Search
• Intelligent Robots: Learning & Optimization for ROB

Application Category

• Search and Retrieval-Augmented AI: Web learning to rank, online learning, and counterfactual learning for ranking
• Graph Algorithms and Modeling for the Web: Foundation models and LLMs for Web-related graphs
• Web Mining and Content Analysis: Robustness and generalizability of Web mining methods

Abstract

Few-shot learning has recently attracted significant interest in drug discovery, with a recent, fast-growing literature mostly involving convoluted meta-learning strategies. We revisit the more straightforward fine-tuning approach for molecular data, and propose a regularized quadratic-probe loss based on the the Mahalanobis distance. We design a dedicated block-coordinate descent optimizer, which avoid the degenerate solutions of our loss. Interestingly, our simple fine-tuning approach achieves highly competitive performances in comparison to state-of-the-art methods, while being applicable to black-box settings and removing the need for specific episodic pre-training strategies. Furthermore, we introduce a new benchmark to assess the robustness of the competing methods to domain shifts. In this setting, our fine-tuning baseline obtains consistently better results than meta-learning methods.

Problem

• Addresses few-shot learning in drug discovery.
• Proposes regularized quadratic-probe loss for molecular data.
• Introduces a new benchmark for domain shift robustness.

Innovation

• Regularized quadratic-probe loss
• Block-coordinate descent optimizer
• Simple fine-tuning approach